International Core Journal of Engineering 2020-26 | Page 129
each target consumer. Finally, we make use of the web
services via front-end browsers to inform consumers with
recommendation via the prediction informing engine.
With the background information, and the comparison
results of the two groups of consumers. For both the A-
group and the B-group, we find out a quantitative method to
contrast: we calculate the matching degree (denoted as M d )
based on the formula below:
M d
R
L
Where R is the matched cellphone numbers of predicted
labels(the recommended cellphone) from training labels(the
actual cellphones of the consumers. And L is the sum of
the training label from the original dataset of cellphones.
The overall cellphone recommendation results and
comparison are shown in Table I.
Fig. 3. Experimental environment.
B. Experimental Procedure
We run four sets of cellphone recommending
mechanisms simultaneously: our proposed method using
Gradient Boosting Decision Tree (GBDT), and Random
Forest (RF) as an alternative method. Others’ method using
Artificial Neural Network (ANN) and Naïve Bayes (NB).
They all meet the basic requirements and business standards
and can output suitable cellphones in a certain degree of
accuracy.
TABLE I. R ECOMMENDATION RESULTS AND COMPARISON
Group
Overall
Consumers
group
A-group
B-group
Recommending method
Existing Old Method
CIMR
Matching degree
90.38%
95.87%
We use Random forest method in B-group and Naïve
Bayes method in A-group to represent to the best
performance for each group. As we can see, the matching
degree for the B-group has increased by 5.49% compared to
the matching degree for the A-group. The considerable
performance fulfills the requirement of consumers definitely.
After applying the recommendation system, consumers
will receive the recommended cellphones from the 16
cellphone models. We then apply the AB testing, consumers
whose cellphones are determined by our methods are called
the B-group and whose cellphones are determined by old
methods are called the A-group. The cellphone
recommendation comparison procedure is as below:
To conduct further analysis of the cellphone
recommendation evaluation, we focus on the distribution of
the 16 original cellphones data from the survey. Fig. 4
displays the distribution of 16 original cellphones. Fig. 5
shows the distribution of the predicted results from CIMR.
Step 1: determination of A-group and B-group of the
target consumers. For each of the two data groups, we both
use all the consumers as the dataset so that the distribution
of the A-group is the same as that the B-group, and we can
avoid some negative effects and random errors.
Step 2: recommending cellphone for every consumer in
the two groups. For all consumers in the dataset, we apply
the user’s portrait as one part of the predicting dataset. Next,
we apply the portrait of recorded 16 cellphones as the other
part. For a target consumer, the user’s portrait will be
matching with each of the 16 cellphones, obtaining 16
results. And the same for the A-group procedure.
Step 3: After obtaining the results. We make a
comparison with the training labels of the target consumers
one by one, and thus we obtain comparison results in the
experimental evaluation. We compare the result from A-
group and that from B-group.
Fig. 4. The distribution of original cellphone data.
C. Experimental Results
We first briefly summarize an overall model prediction
result considering the scores and variation in evaluating the
4 models. Averagely, RF prevails in all score testing:
precision, recall and F1-score, and with least variation in
predicting each of the 16 cellphones. GBDT takes up the
second place in both scores testing and variation stability.
ANN becomes the third in the scores testing and the fourth
in the variation stability. Finally, NB performs last in testing
the score and takes the third place in variation stability.
Fig. 5. The distribution of predicted cellphone data.
107