International Core Journal of Engineering 2020-26 | Page 129

each target consumer. Finally, we make use of the web services via front-end browsers to inform consumers with recommendation via the prediction informing engine. With the background information, and the comparison results of the two groups of consumers. For both the A- group and the B-group, we find out a quantitative method to contrast: we calculate the matching degree (denoted as M d ) based on the formula below: M d R L Where R is the matched cellphone numbers of predicted labels(the recommended cellphone) from training labels(the actual cellphones of the consumers . And L is the sum of the training label from the original dataset of cellphones. The overall cellphone recommendation results and comparison are shown in Table I. Fig. 3. Experimental environment. B. Experimental Procedure We run four sets of cellphone recommending mechanisms simultaneously: our proposed method using Gradient Boosting Decision Tree (GBDT), and Random Forest (RF) as an alternative method. Others’ method using Artificial Neural Network (ANN) and Naïve Bayes (NB). They all meet the basic requirements and business standards and can output suitable cellphones in a certain degree of accuracy. TABLE I. R ECOMMENDATION RESULTS AND COMPARISON Group Overall Consumers group A-group B-group Recommending method Existing Old Method CIMR Matching degree 90.38% 95.87% We use Random forest method in B-group and Naïve Bayes method in A-group to represent to the best performance for each group. As we can see, the matching degree for the B-group has increased by 5.49% compared to the matching degree for the A-group. The considerable performance fulfills the requirement of consumers definitely. After applying the recommendation system, consumers will receive the recommended cellphones from the 16 cellphone models. We then apply the AB testing, consumers whose cellphones are determined by our methods are called the B-group and whose cellphones are determined by old methods are called the A-group. The cellphone recommendation comparison procedure is as below: To conduct further analysis of the cellphone recommendation evaluation, we focus on the distribution of the 16 original cellphones data from the survey. Fig. 4 displays the distribution of 16 original cellphones. Fig. 5 shows the distribution of the predicted results from CIMR. Step 1: determination of A-group and B-group of the target consumers. For each of the two data groups, we both use all the consumers as the dataset so that the distribution of the A-group is the same as that the B-group, and we can avoid some negative effects and random errors. Step 2: recommending cellphone for every consumer in the two groups. For all consumers in the dataset, we apply the user’s portrait as one part of the predicting dataset. Next, we apply the portrait of recorded 16 cellphones as the other part. For a target consumer, the user’s portrait will be matching with each of the 16 cellphones, obtaining 16 results. And the same for the A-group procedure. Step 3: After obtaining the results. We make a comparison with the training labels of the target consumers one by one, and thus we obtain comparison results in the experimental evaluation. We compare the result from A- group and that from B-group. Fig. 4. The distribution of original cellphone data. C. Experimental Results We first briefly summarize an overall model prediction result considering the scores and variation in evaluating the 4 models. Averagely, RF prevails in all score testing: precision, recall and F1-score, and with least variation in predicting each of the 16 cellphones. GBDT takes up the second place in both scores testing and variation stability. ANN becomes the third in the scores testing and the fourth in the variation stability. Finally, NB performs last in testing the score and takes the third place in variation stability. Fig. 5. The distribution of predicted cellphone data. 107