Measurement and Evaluation of the Online Marketing Training Results for Students at SMA Negeri 1 Bantur Malang

SMA Negeri 1 Bantur Malang is located in the southern area of Malang Regency. The demographic conditions and the Bantur community characteristics have caused many SMA Negeri 1 Bantur graduates to choose to work rather than continue their education to higher education. As a solution to this condition, the school has provided job training to students for one year as a provision for them after graduation. Apart from being in the form of student knowledge and skills, job training activities also produce goods and services that can be marketed to the public. Therefore, after participating in the training activities, SMA Negeri 1 Bantur students need to also be given training in marketing goods and services. This paper presents the results of online marketing training for goods and services at SMAN 1 Bantur. The method used in the training is tutorial and practice using text and video media. Measuring the success of training activities using the questionnaire method and assessment of the training participants' practice results. The results of measuring the success of training and their evaluation can measure the increase in knowledge and skills of each student who participates in the training.

Apart from being in the form of knowledge and skills, job training activities also produce goods and services. These goods and services are the work of students during job training. Some goods and services are even feasible enough to be marketed to the public. Some students who have completed training have even been able to produce goods or services that are suitable for marketing (Bustomi, et al, 2020). In connection with the marketing of goods and services from job training, problems arise, because most students do not yet have the knowledge and skills about marketing goods and services. As a solution to this problem, SMA Negeri 1 Bantur students need to be given training in the marketing of goods and services after they have attended job training. We have carried out training activities for marketing goods and services for SMA Negeri 1 Bantur as part of community service activities. In the training, students are introduced to marketing concepts and strategies and the use of several online marketing applications such as shopee and tokopedia. After attending the training, students are expected to be able to market their products to consumers through online marketing applications (Bustomi, et al, 2020). This paper describes the results of measurement and evaluation of training activities for the marketing of goods and services online at SMA Negeri 1 Bantur. Measurement of the results of training activities is carried out in two ways, namely the questionnaire method and the method of assessing the training participants' practical results. The research objective was to obtain a measure of the success of online marketing training activities at SMA Negeri 1 Bantur. The theoretical framework used in the study refers to the evaluation model developed by Peltier et al. on online marketing training (Peltier, Drago, and Schibrowsky, 2003), D. L. Kirkpatric and J. D. Kirkpatrick on how to evaluate a training program (Kirkpatric and Kirkpatrick, 2009), and Ritzmann et al. about designing a training evaluation (Ritzmann, Hagemann, and Kluge, 2014). The essence of the theoretical framework used by the three research groups states that evaluating the success of a training program must pay attention to the availability of training facilities, the quality of human resources involved in the training, and the training methods used. Departing from the core of the theoretical framework, the preparation of training evaluations in the form of qualifying designs and practical assessment points will pay attention to these aspects.
The novelty of this research is that the method of measuring the success of the training is a combination of the results of the questionnaire and the assessment of the training participants' practice results. The combination of the two methods is necessary to obtain the optimum conditions from the two evaluation methods which have opposite characteristics. The questionnaire method can measure large numbers of parameters quickly but has low accuracy of results. In contrast, the assessment of the training process can measure with high accuracy, but it takes a long time. The results and benefits that can be obtained from this research are knowledge about the appropriate combination of evaluation between the questionnaire method and the assessment of the training process to be able to provide optimal conditions between the number of parameters that can be extracted and the accuracy of the measurement results.

Work Training
SMA Negeri 1 Bantur is located in the southern part of the Malang Regency. Most of the people in the southern region of the Malang Regency are people who have traditional views. The factor of the Bantur people's view of life is quite influential on graduates of SMA Negeri 1 Bantur. Most of the graduates of SMA Negeri 1 Bantur did not go to university and chose to work in the southern part of the Malang Regency. The southern region of the Malang Regency is a hilly area facing the southern coast of Java which is famous for its big waves of the Indian Ocean. Most of the people in the southern region of the Malang Regency work as traditional farmers who rely on rice fields, cultivation, or plantations. The demographic conditions and characteristics of the Bantur community that are still traditional cause many of their children who have graduated from high school to choose to work rather than continue to higher education levels.
Likewise, what happened at SMA Negeri 1 Bantur, many students chose to work after they graduated (Bustomi, et al, 2019).
As a solution to this condition, the school has provided job training to students as provisions after graduation. This job training is a program of the East Java provincial government which is given to several SMA or MA whose graduates prefer to work rather than continue to university. In this job training, students are given choices of skill areas according to school proposals and student interests. The job training is given for one year when the students are in class XI.

Online Marketing Training
Online marketing has several advantages over conventional marketing. Some of these advantages are that online marketing is cheaper in terms of costs, online marketing is easier to do in terms of procedures, online marketing is wider in area coverage in terms of area, and other advantages. Online marketing provides opportunities for anyone to market goods and services easily and inexpensively (Bostanshirin, 2014, Lidiawaty, Irawan, andGinardi, 2020).
Various advantages of online marketing need to be utilized to help small business community groups in marketing their goods and services. By taking advantage of online marketing, small business people will have the opportunity to develop. These small business actors need to be given training in the marketing of goods and services online first. The training activity is an effort to introduce online marketing to small entrepreneurs (Sulistyowati, et al, 2018, Hartono and Ubed, 2018, Hidayah, et al, 2019.

Marketing Education
The development of information and communication technology has made the field of marketing undergo rapid changes. Marketing of goods and services that was previously carried out traditionally can now be done easily using the internet media. Internet media has opened up new, wider markets for the marketing of goods and services (Brocato, et al, 2015, Evans, 2001, Finch, Nadeau, and O'Reilly, 2012.
Various advantages of online marketing have attracted many people from all over the world to develop and use it. Various methods, procedures, and online marketing applications have been developed.
This development is also triggered by the desire to build a global market that is no longer bound by national boundaries (Close, Dixit, and Malhotra, 2005, Granitz and Koernig, 2011, Khan, 2013. This development also has an impact on changes in the marketing education system which must adapt to developments in information and communication technology. Various changes have been made in various educational institutions in many countries to respond to developments in information and communication technology in the marketing field. Teaching concepts and methods, as well as the marketing learning curriculum, must be adapted to the latest developments. Peltier and colleagues discuss the relationship between virtual communities and online marketing education assessments (Peltier, Drago, and Schibrowsky, 2003). Peterson and colleagues discuss the instructional use of information and communication technology in marketing education (Peterson, et al, 2002). Shaltoni discusses the transition process of online marketing education in a review of international courses and programs (Shaltoni, 2016).

Evaluation of Learning Outcomes
In the learning process such as training, lectures, and others there must be an evaluation of the learning process. The evaluation aims to get feedback so that quality increases in the next learning process.
By implementing a good evaluation in the learning process, it will also get better feedback to improve the quality of the next learning process. There are various methods for evaluating a process. One example is the evaluation method using the RFM model and clustering to evaluate customer segments (Rahadian and Syairudin, 2020). In subsequent research, evaluation methods using the RFM model and clustering also need to be accommodated and applied to the learning process.

Training Method
The method used in training is a combination of tutorials and hands-on practice. The media used in the tutorial are text and video shows. Text media in the form of ppt and pdf files and video media in the form of the video shows from YouTube links. Both types of training media were given to participants a few days before the implementation of the training activities. The main material is training on the use of online marketing applications to market goods and services. Online marketing applications used in training activities are Shopee and Tokopedia. Both applications were chosen because they were well known to the public.

Measurement of Training Success
To determine the success of training activities, a method of measuring the process and results of training is needed. Measuring the success of the training is carried out to get feedback for evaluation and improvement of the quality of the training in the future. In this training activity, two types of success measurement methods are used, namely the questionnaire method and the method of assessing the results of the participants' practice. Both methods are designed to obtain quantitative measurement results.
Therefore, each participant's answer to the questionnaire and the participant's practice results will be converted into numbers. The conversion in numbers is carried out based on the conversion system design that has been prepared before the implementation of the training.
The questionnaire method was used to map the condition of each participant's knowledge of online marketing before, during, and after the training. The training results are assessed by the trainers during the training. For the assessment of practical results by each participant, there are three categories, namely fluency, need help, and not yet successful.

Population and Research Sampling
This research and community service activity is carried out for the younger generation of the Bantur sub-district community who have attended job training. These young people are in the age range of 17-19 years. For research purposes related to measuring the results of online marketing training, 12 of these youth were selected according to the recommendations of SMA Negeri 1 Bantur based on the representation of each group in the field of work skills. Thus, the study population was the youth of the Bantur sub-district who had completed job training. The research sample was 12 participants of online marketing training who had been selected from the youth of the Bantur sub-district.

The questionnaire Design
The questionnaire was designed to follow the evaluation design concept in a training given by Peltier et al. (Peltier, Drago, and Schibrowsky, 2003), D. L. Kirkpatric and J. D. Kirkpatrick (Kirkpatric and Kirkpatrick, 2009), and Ritzmann et al. (Ritzmann, Hagemann, and Kluge, 2014). Starting from the concept of evaluation design in a training activity and adapting it to the conditions of the implementation of the training activity, an analysis tool for measuring the success of the training is compiled. There are two kinds of adjustments that need to be made, namely adjustments related to the combination of assessment and questionnaire results and adjustments related to the weighting of the participant's motivation parameters which are higher than the parameters of mastery of the training material. The first adjustment regarding the combination of the questionnaire and the assessment process is carried out to obtain the optimum conditions for the two methods which have opposite characteristics. The second adjustment relates to the weighting of the parameters that are not uniformly carried out to synchronize the evaluation with the main objective of the training which focuses more on sharing knowledge from participants to other community members.
The questionnaire was designed in three parts. The first part was used to gain knowledge of each trainee about various online marketing applications before they took part in the training (Table 1). The second part is used to determine the absorption capacity of each participant during the training activities (Table 2). The third part explores the motivation of each participant to share knowledge and skills with others after they have completed the training (Table 3). Have you ever used an online marketing application?
A not ever 0 B Ever 50 C Often 100 5 15% According to you, is good where marketers an unusual or marketing online?
A Usual marketing 0 B Online marketing 100 C same course 50 In Table 1, Table 2, and Table 3 each question has its percentage weight. Also, each participant's choice of answers has its score points. There are only three points for the answer choices, namely 0, 50, and 100.

Assessment of the Participants' Practice
Assessment of the results of the participants' practice is based on three categories, namely fluency, need help, and not yet successful. The fluency category is given to participants if the trainees can practice online marketing applications quite fluently. Categories need assistance given to participants if trainees can practice online marketing applications after getting directions and assistance from trainers. The category has not been successfully awarded to participants if participants have not succeeded in practicing online marketing applications even though they have received direction and assistance from trainers.

Measuring Method of Training Results
The stages of the training success measurement method are shown in Figure 1. There are four stages in the measurement method, namely collecting data from each participant, determining the value of each participant's questionnaire, determining the value of the training results for each participant, and determining the success value of each participant's training. The first stage of collecting data from each participant was carried out in two ways, namely through the questions in the questionnaire and the assessment of the results of the practice of each participant by the assessment of the results of the participants' practice by the trainer (given the symbol P). The questions in the questionnaire consisted of three parts, namely questions to measure participants 'initial knowledge (given the symbol IK), questions to measure the participants' learning absorption (given the symbol LA), and questions to measure the motivation to share skills (given the symbol WS).
The second stage of determining the value of the questionnaire (given the symbol Q) is based on the recognition of participants with a greater emphasis on the willingness to share (WS) aspect of 60% than on the aspect of the ability to absorb learning materials (LA) of 40%. This refers to the objectives of the training, namely that participants are expected to pass on their knowledge and skills to the community. So that the value of Q can be expressed as in Equation (1). (1) The third stage of determining the value of the training results (given the symbol T) for each participant is obtained from the Q questionnaire value and the P-value of each participant's practice. The training result value (T) is a balanced combination between the questionnaire value of each participant (Q) of 50% and the practice value of each participant (P) of 50%. So that the value of T can be expressed as in Equation (2).
The fourth or final stage is to determine the value of the success of the training (given the symbol S) for each participant. The success value of the training is based on increasing the knowledge or skills of each participant before training (IK) and after training (T). So that the value of S can be expressed as in Equation (3).

S = T -IK (3)
The stages in this research can be stated in the form of a flow chart. The shape of the research flow chart is shown in Figure 1.

Participants' Questionnaire Answers
The first stage of the research was collecting data from each participant. The data collection method is a questionnaire and a practical assessment of each training participant. The results of the questionnaire from each participant are shown in Table 4.   Question  Model  PARTICIPANTS ' NUMBER  01  02  03  04  05  06  07 08  09  10  11  12  11 B

Participants' Willingness to Share after the Training
In Table 4, the results of the answers to all 12 training participants are presented. Each participant gets a participant code in the form of a two-digit number from 01 to 12. Table 4  the same as the model's answer, then the success of the participant will get the category as expected. If the score of the participant's answer is higher than the score for the model's answer, then the success of the participant is categorized as above expectations.

Participant Value Processing Results
From the participants 'answers in Table 4 and the list of questionnaire answers in Table 1, Table 2, and Table 3, it is possible to convert participants' answers into numbers. Based on the questionnaire design in Table 1, the value of each participant's initial knowledge (IK) can be calculated. Next, based on the questionnaire design in Table 2, the value of the learning material absorption of each participant (LA) can be calculated. And finally, based on the questionnaire design in Table 3, the value of the motivation to share knowledge of each participant (WS) can be calculated. In Table 5, the results of the processing of these data are presented. Also, in the last row of Table 5, the results of each participant's practice are presented. Table   5 also presents the values obtained by the model as a comparison.

Value of Success
The second stage of the study was to determine the value of each participant's questionnaire. From Table 5, the values of IK, LA, WS, and P of each training participant have been presented. The questionnaire value (Q) of each participant is calculated using Equation (1): The third stage is to determine the value of the training results for each participant. From the Q value calculated in the second stage of the study and the P-value in Table 5, it can be calculated that the training result (T) value of each participant using Equation (2): The fourth or final stage is to determine the value of the success of each training participant. The training success value (S) of each participant is obtained by subtracting the T value obtained in the third stage of the study against the IK value in Table 5. The success value S is calculated using Equation (3): In Table 6, a recapitulation of the calculation of the value of training results (T) is presented in the second stage of the study, rewriting of the initial knowledge value (IK) from Table 5 in the first stage of the study, and the training success value (S) in the fourth stage of the study.

Discussion
In Table 6 the success values, the red color indicates the success values that are below expectations.
Three trainees had success scores below expectations. The percentage of participants with scores below expectations is 3/12 or 25%. The orange color represents the success value as expected. Two training participants have the success value as expected. The percentage of participants who scored as expected was 2/12 or 17%. The green color represents success values that are above expectations. Seven training participants had success scores above expectations. The percentage of participants with scores above expectations was 7/12 or 56%.
The results of the measurement and evaluation of this training activity provide better results than the results of the measurement and evaluation of the training activities at SMA Negeri Bendungan Trenggalek.
This is because the results of the evaluation of the training activities at SMA Negeri Bendungan Trenggalek are entirely based on the results of the questionnaire alone (Bustomi, et al, 2019). So with a combination of measuring the results of the training between the questionnaire and the participant's practice assessment, the results are better than the results of the questionnaire alone.
Measuring and evaluating training activities using the questionnaire method is a practical and simple option compared to other measurement methods. Therefore, this method has been widely chosen as a way of obtaining feedback for activity evaluation materials (Kirkpatric and Kirkpatrick, 2009) (Ritzmann, Hagemann, and Kluge, 2014). But the questionnaire method has several weaknesses that stem from the honesty level of the participants who filled out the questionnaire. Therefore, in this study, training measurement and evaluation is a combination of a questionnaire and an assessment of the training process.
So the measurement and evaluation of training do not rely solely on the results of the questionnaire. The combination of measurement between the questionnaire and the assessment of the training process gives better results than the measurement based on the questionnaire alone.
An important contribution obtained from this research is that the evaluation of training results using a combination of measurements between questionnaires and assessment of the training process is an appropriate compromise between two opposing things, namely the number of assessment parameters and the accuracy of the assessment results. Many assessment parameters can be obtained quickly when using the questionnaire method which has a weakness in the accuracy of the results. Conversely, direct assessment of the training process can be obtained with accurate results, but it has weaknesses in the long measurement process. With the combination of the two methods, it is hoped that an optimal condition can be obtained between the number of parameters that can be extracted and the accuracy of the measurement results.
The results of the measurement and evaluation of this training activity can be further developed for further training activities. For example, development was carried out by adopting the model used by Rahadian and Syairudin in the segmentation analysis of course participants using the RFM and clustering model (Rahadian and Syairudin, 2020). The use of the RFM and clustering models is expected to provide a better evaluation of the training participant segmentation. With a good evaluation of participant segmentation, training participants will be obtained who are appropriate and right on target with the training topic. So that the effectiveness of the implementation of the training and the value of its success will be more optimal.

Conclusion
The results of measuring the success of training and their evaluation can measure the increase in knowledge and skills of each student who participates in the training. Measurement of the success of the training that has been carried out obtained the following results: 25 % of participants had success values below expectations, 17 % of participants had success values as expected, and 58 % of participants had success values that exceeded expectations. The combination of measurement between the questionnaire and the assessment of the training process applied in this study gave better results than the measurement based on the questionnaire alone applied in previous studies. Evaluation of training results using a combination of measurements between questionnaires and assessment of the training process is an appropriate compromise between the number of assessment parameters and the accuracy of the assessment results.
Further development of measurement and evaluation methods for training can be carried out by adopting the RFM and clustering models. The use of the RFM and clustering models is expected to provide a better evaluation of the training participant segmentation so that training participants can be obtained that are appropriate and right on target with the training topic.