Utilizing our global community database of more than 1.3 million contributors, DataForce quickly sourced and trained a team of 11 with the relevant skills and background in a two-week timeframe. A dedicated project manager and coordinator were assigned to ensure the timeline, training, budget, and all other particulars were met along with seven annotators and two reviewers for the classification portion. Each tweet was assigned through our proprietary annotation platform to two different annotators to ensure there were multiple perceptions per tweet. In the case of a disagreement, the results were then allocated to a QA reviewer. The QA reviewer would disambiguate the initial categorizations and decipher the tweet alongside the annotator’s viewpoint to determine categorical placement based on guidelines received by our client.
Given the high subjectivity of the task, we made sure to be on target by categorizing and delivering a fraction of the dataset first, which included 250 tweets. After completing the pilot and discussing the outcome with the client, we further fine-tuned the guidelines and specified training instructions before going into the full production of 1,926 tweets. It is important to note that this review of the first 250 tweets provided a mutual opportunity to dissect the guidelines and reassess the categorization breakdown, providing an opportunity to alter the project early on so our client could obtain the results they desired in the end.
Our client not only received the final data annotated within eight weeks of project kickoff, but they also had the ability to view the results in a simple way, thanks to a customized dashboard we built catered to their preferences.
Following the final delivery, we received excellent feedback from our client noting the measurable shift in the performance of their model thanks to the labeled and classified dataset. We were proud to report a 100% acceptance score, indicating our client was fully satisfied with the quality of the dataset results.