Labeling Kenyan Social Media Data for a Sentiment Analysis Tool

UNDP works in 170 countries and territories to eradicate poverty while protecting the planet. We help countries develop strong policies, skills, partnerships and institutions so they can sustain their progress The primary and overarching objective of United Nations Development Programme in Kenya is the eradication of poverty in the context of sustainable development, including the pursuit of the Sustainable Development Goals, and promotion of United Nations fundamental principles. A core dimension to the work of UNDP in Kenya is on Democratic Governance given the national focus on governance reforms. UNDP supports the country’s efforts towards achieving the Vision 2030 Political Pillar, which envisions a democratic system that is issue-based, people-centered, results oriented and accountable to the public. This Political Pillar gears to transform the country’s political governance across five strategic areas; The Rule of Law, Electoral and Political Processes, Democracy and Public Service Delivery, Transparency and Accountability, Security Peace Building and Conflict Management. These strategic areas are anchored in the Constitution, promulgated in August 2010 marking a major milestone in the democratic journey of Kenya and set a new threshold in terms of people-centred development.

The Regional Centre of Competence (RCoC) for Digital and AI Skilling — a partnership between the Government of Kenya, Microsoft and the Kenya School of Government, supported by UNDP Kenya — is developing a Sentiment Analysis tool adapted to Kenya's multilingual digital landscape. Standard sentiment models are trained largely on structured, global-English datasets and miss the meaning carried by local expression, code-switching and slang. To close this gap, the RCoC is fine-tuning the XLM-RoBERTa base model on Kenyan social media data so that the tool can read sentiment and emotion accurately across English, Kiswahili and Sheng. This work requires a high-quality, human-labeled dataset to serve as the ground-truth training data for the tool. Online Volunteers will produce this dataset. Through the assignment, volunteers gain practical, portfolio-ready experience in natural language processing (NLP) data annotation, inter-annotator quality methods, and applied AI for the public good, while contributing directly to a locally relevant and responsibly built AI tool for the Government of Kenya. The assignment advances SDG 9 by strengthening innovation and digital capabilities suited to local needs. Volunteers will receive a certificate of appreciation upon successful completion.

We are looking for four (4) Online Volunteers to annotate a dataset of Kenyan social media text so it can be used as ground-truth training data for the RCoC's Sentiment Analysis tool. The dataset include 10,000 data rows and will be cross-checked to verify accuracy and ensure high confidence in the data. The assignment is carried out under the overall supervision of the RCoC Project Lead, while day-to-day guidance on the annotation task is provided by the Junior AI Solution Developers under the RCoC. Each Online Volunteer will: - Attend an initial onboarding session and complete a test run of 50–100 rows to align on the guidelines and category definitions before the annotation period begins. - Independently annotate the complete dataset of 10,000 rows, performing: - Sentiment Annotation: categorizing the overall sentiment of each text snippet into the designated categories (e.g., Positive, Negative, Neutral). - Emotion Classification: labeling the specific emotion conveyed in the text based on the predefined set of categories (e.g., Joy, Anger, Sadness, Fear, Disgust, Surprise). - Contextual Interpretation: accurately interpreting and tagging slang, code-switching and context-specific phrasing heavily used in Kenyan digital spaces (specifically English, Kiswahili and Sheng blends). - Participate in quality assurance through overlap-labeling: Inter-Annotator Agreement (IAA) is measured across every row to ensure high confidence in the data quality. - Conduct all labeling directly within Label Studio (for web-based collaborative tagging) or Microsoft Excel (for offline batch processing), as directed by the project team, following the provided annotation guidelines and category definitions. Deliverables (per Online Volunteer) 1. Completed calibration test run of 50–100 rows and confirmed alignment on the annotation guidelines and category definitions. 2. One complete and accurately annotated set of all 10,000 rows, labeled for sentiment and emotion in adherence with the provided annotation guidelines and category definitions. 3. A complete data export from Label Studio, or an updated Microsoft Excel sheet, fully formatted according to the project's data schema. Workload: at 16–20 hours per week over the 4-week period, each volunteer independently annotates the full 10,000-row dataset.