In this short article we report the results of the experiment created to test the ability of Tapoi’s profiling and modelling technology to actually predict the interests of users.
Tapoi is an innovative customer intelligence service for web portals and mobile applications. It gives actionable knowledge on the interests and attitudes of online visitors. Businesses can leverage this information in order to provide targeted and tailored services, personalize the user experience and obtain higher conversion.
Tapoi mines in user activities on the most popular social media platforms (Facebook, Twitter) to dynamically build rich user profiles. State-of-the-art semantic analysis technologies are used to build a robust and reliable map of users' topical attitudes, interests and preferences.
The on-field trial experiment has involved more than 150 users and has been run in collaboration with EIT Digital , a leading European digital innovation and entrepreneurial education organization, and SmartCrowds , a Territorial Lab located in Trento (Italy) that aims to involve citizens in technological research and innovation. Users were asked to log in using their Facebook account and were presented with their profile as built by Tapoi. They were then guided through a survey to assess whether they thought the Tapoi profile was accurate and in line with their actual interests.
The objectives of the experiment were to:
Tapoi profiles are based on models. Formally, a model is a computation that takes as inputs the actions performed by users (a post, like, share etc.) and outputs a set of relevant quantitative indicators.
Concretely, a model allows Tapoi to measure the interests demonstrated by a user in a particular vertical domain.
The three models considered were:
The experiment consisted of three main steps:
During this step users were asked to rate their personal interests (the real interests as a person) and the social ones (the interests demonstrated on Facebook) according to the categories of each model. For each dimension the user could express a value in the range 1 (lowest) to 10 (maximum).
Users were then presented with a summary of the chosen information, together with their profile as computed by Tapoi. This comparison was represented with a spider chart, as it can be seen in the figure below. Next to the chart, a word cloud collects the entities that Tapoi recognized from the mining of the user’s online activities.!
As a last step, users were asked to rate the perceived accuracy in the estimation of their interests and they optionally had the chance to leave a textual feedback.
User had also the possibility to point out errors Tapoi had made during the analysis by selecting and deleting words that:
This step was of key importance for improving the modeling capabilities of Tapoi.
The experiment was up and running for 7 days and involved 161 users (84 male and 77 female). Their feedback allowed us to collect information about the general satisfaction in the profiling and modelling accuracy. The values of user satisfactions have been computed for each model by taking into consideration the direct feedbacks given by the users with a value ranging from 1 to 10.
Tapoi totalized an average 63% of satisfaction rating, further classified as it follows:
The objective accuracy was computed by comparing the rating of users' social interests collected in the first phase to the rating calculated by Tapoi. With an average of 73%, the model accuracy was divided as follows among the three models:
The experiment allowed us to gain very useful insights into the perception by users of Tapoi-generated profiles. Furthermore, the experiment results proved that Tapoi is able to characterize user’s interests with a high level of precision and can be purposefully used for tailoring online user experience and improving the engagement and loyalty of online users.