Analysis of Policy Agendas: Lessons Learned from Automatic Topic Classification of Croatian Political Texts

Policy agenda research is concerned with measuring the policymaker activities. Topic classification has proven a valuable tool for policy agenda research. However, manual topic coding is extremely costly and time-consuming. Supervised topic classification offers a cost- effective and reliable alternative, yet it introduces new challenges, the most significant of which are the training set coding, classifier design, and accuracy-efficiency trade-off. In this work, we address these challenges in the context of the recently launched Croatian Policy Agendas project. We describe a new policy agenda dataset, explore the many system design choices, and report on the in- sights gained. Our best-performing model reaches 77% and 68% of F1- score for ma- jor topics and subtopics, respectively.