Quarterly Journal of Governance Knowledge

Quarterly Journal of Governance Knowledge

Investigating the Role of Data Mining in Enhancing Science and Technology Policymaking: Analyzing Functions and Implications

Document Type : Original Article

Authors
Leila Namdarian 1
Hamid Reza Khedmatgozar 2
1 Associate Professor, Information and Society Research Department, Iranian Research Institute for Information Science and Technology (IRANDOC), Tehran, Iran
2 Assistant Professor, Information Technology Research Department, Iranian Research Institute for Information Science and Technology (IRANDOC), Tehran, Iran
10.22034/jokog.2024.481554.1019
Abstract
In the digital age and today’s complex, fast-paced world, policymaking in science, technology, and innovation faces numerous challenges. Rapid technological advancements, diverse societal needs, and the vast volume of data have made policymaking processes increasingly intricate. In this context, data mining has emerged as a novel and powerful tool, offering innovative solutions for analyzing, forecasting, and evaluating policies. By extracting valuable insights from big data, data mining enables policymakers to make more effective decisions grounded in precise analysis and evidence-based forecasts. These capabilities are particularly vital in the realm of science and technology, where the complexity of information and the presence of nonlinear relationships are prominent.
This study primarily aims to identify the roles and functions of data mining in science and technology policymaking. It endeavors to provide a framework for the practical utilization of data mining throughout the policymaking process, from problem identification and definition to policy impact evaluation.
Methodologically, the study employs a systematic literature review to gather data, thematic analysis to identify patterns and key concepts, and constant comparative analysis to develop a comprehensive framework. The analyzed sources include scientific articles, policy reports, and case studies focusing specifically on data mining applications in science and technology policymaking. Through a structured analysis process, the study extracts and categorizes the roles and functions of data mining across various aspects of policymaking.
Key findings indicate that data mining serves five pivotal functions in science and technology policymaking:

Trend analysis and future forecasting: Utilizing machine learning algorithms and historical data analysis, data mining uncovers hidden patterns and predicts shifts in science and technology priorities. This capability prepares policymakers for an uncertain future.
Policy impact assessment and effectiveness evaluation: By analyzing the outcomes of implemented policies, data mining provides precise feedback to enhance and refine policymaking strategies.
Risk prediction and management: Data mining identifies risk factors and offers preventive analyses, enabling policymakers to manage potential risks and mitigate adverse consequences.
Resource prioritization and optimization: Through big data analysis, data mining identifies high-impact areas and optimizes resource allocation to these domains.
Decision support through scenario evaluation: Data mining assists policymakers in evaluating alternative scenarios and selecting the most effective options based on detailed data-driven insights.

The findings underscore the transformative role of data mining in science and technology policymaking. With advanced data mining tools, policymakers can make informed, evidence-based decisions in complex and dynamic environments. Analyzing extensive data from diverse sources facilitates the identification of emerging challenges and opportunities, contributing to the development of flexible and efficient policies. Moreover, the study demonstrates that data mining not only enhances decision-making processes but also fosters transparency and accountability in policymaking.
Practical and academic implications of this study include:

Facilitating evidence-based decision-making across all stages of policymaking.
Identifying emerging technologies and formulating supportive policies to promote innovation.
Improving resource management and minimizing wastage through optimized allocations.
Enhancing transparency and accountability in policymaking through precise data analysis.
Providing an operational framework for leveraging data mining to evaluate and improve policy strategies.

This study provides a new perspective on the application of data mining in science and technology policymaking by identifying and analyzing its roles and functions. Data mining enables policymakers to navigate uncertainties, achieving more precise and effective outcomes. The adoption of data mining not only improves policymaking in science and technology but also serves as a model for other policy domains.
Keywords
Data Mining in Policymaking
Trend Analysis
Risk Management
Resource Allocation Optimization
Policy Effectiveness Assessment
Alternative Scenario Evaluation
Subjects
Aggarwal, C. C. (2015). Data mining: the textbook (Vol. 1, p. 1). New York: springer.
Alharbi, E., Skeva, R., Juty, N., Jay, C., & Goble, C. (2023). A FAIR-Decide framework for pharmaceutical R&D: FAIR data cost–benefit assessment. Drug discovery today, 28(4), 103510. doi: 10.1016/j.drudis.2023.103510
Ali, A., Hussain, T., Tantashutikun, N., Hussain, N., & Cocetta, G. (2023). Application of smart techniques, internet of things and data mining for resource use efficient and sustainable crop production. Agriculture, 13(2), 397. doi: 10.3390/agriculture13020397
Androutsopoulou, A., & Charalabidis, Y. (2018, April). A framework for evidence-based policy making combining big data, dynamic modelling and machine intelligence. In Proceedings of the 11th International Conference on Theory and Practice of Electronic Governance (pp. 575-583).
Androutsopoulou, A., & Charalabidis, Y. (2021). A model for evidence-based social policy making, driven by Big Data, dynamic simulation and stakeholders participation. In Perspectives for digital social innovation to reshape the European welfare systems (pp. 202-218). IOS Press. doi: 10.3233/STPC200012
Ang, K. L. M., Seng, J. K. P., Ngharamike, E., & Ijemaru, G. K. (2022). Emerging technologies for smart cities’ transportation: geo-information, data analytics and machine learning approaches. ISPRS International Journal of Geo-Information, 11(2), 85. doi: 10.3390/ijgi11020085
Araz, O. M., Choi, T. M., Olson, D. L., & Salman, F. S. (2020). Role of analytics for operational risk management in the era of big data. Decision Sciences, 51(6), 1320-1346. doi: 10.1111/deci.12451
Arfanuzzaman, M. (2021). Harnessing artificial intelligence and big data for SDGs and prosperous urban future in South Asia. Environmental and sustainability indicators, 11, 100127. doi: 10.1016/j.indic.2021.100127
Ashford, L. S., Smith, R. R., De Souza, R. M., Fikree, F. F., & Yinger, N. V. (2006). Creating windows of opportunity for policy change: incorporating evidence into decentralized planning in Kenya. Bulletin of the World Health Organization, 84(8), 669-672. doi: 10.2471/BLT.06.030593
Athey, S., & Imbens, G. W. (2017). The state of applied econometrics: Causality and policy evaluation. Journal of Economic perspectives, 31(2), 3-32. doi: 10.1257/jep.31.2.3
Babaiyan, Fatemeh, Safdari Ranjbar, Mustafa, Hakim, Amin. (1402). Analyzing the role of artificial intelligence in the public policy cycle; a meta-synthesis approach. Management Improvement, 17 (2), 115-150.[In Persian].doi: 10.22034/jmi.2023.396945.2957[link]
Baeza-Yates, R., & Ribeiro-Neto, B. (1999). Modern information retrieval (Vol. 463, No. 1999). New York: ACM press.
Bengisu, M., & Nekhili, R. (2006). Forecasting emerging technologies with the aid of science and technology databases. Technological Forecasting and Social Change, 73(7), 835-844. doi: 10.1016/j.techfore.2005.09.001
Boeije H. (2002). A Purposeful Approach to the Constant Comparative Method in the Analysis of Qualitative Interviews, Quality & Quantity, 36, 391–409. doi: 10.1023/A:1020909529486
Braun, V., & Clarke, V. (2023). Toward good practice in thematic analysis: Avoiding common problems and be (com) ing a knowing researcher. International journal of transgender health, 24(1), 1-6. doi: 10.1080/26895269.2022.2129597
Cairney, P., & Oliver, K. (2017). Evidence-based policymaking is not like evidence-based medicine, so how far should you go to bridge the divide between evidence and policy? Health research policy and systems, 15, 1-11. doi: 10.1186/s12961-017-0192-x
Chy, M. B. O. N., & Buadi, O. N. (2024). Role of Machine Learning in Policy Making and Evaluation. International Journal of Innovative Science and Research Technology, 9(10), 456-463. doi: 10.38124/ijisrt/IJISRT24OCT687
Delen, D., & Demirkan, H. (2013). Data, information and analytics as services. Decision Support Systems, 55(1), 359-363. doi: 10.1016/j.dss.2012.05.044
Dewantoro, T. B., Nugroho, L. E., & Permanasari, A. E. (2024, August). Implementing Crowdsourcing in Smart Government: An IT Perspective Review. In 2024 International Electronics Symposium (IES) (pp. 329-334). IEEE. doi: 10.1109/IES63037.2024.10665848
Domingues, M. S., Baptista, A. L., & Diogo, M. T. (2017). Engineering complex systems applied to risk management in the mining industry. International journal of mining science and technology, 27(4), 611-616. doi: 10.1016/j.ijmst.2017.05.007
Edwards, M. (2005). Social science research and public policy: narrowing the divide. Australian Journal of Public Administration, 64(1), 68-74. doi: 10.1111/j.1467-8500.2005.00417.x
Fan, W., & Bifet, A. (2013). Mining big data: current status and forecast to the future. ACM SIGKDD explorations newsletter, 14(2), 1-5. doi: 10.1145/2481244.2481246
Fayyad, U., Piatetsky-Shapiro, G., & Smyth, P. (1996). From data mining to knowledge discovery in databases. AI magazine, 17(3), 37-37. doi: 10.1609/aimag.v17i3.1230
Gokhberg, L. (2020). Use AI to mine literature for policymaking. Nature, 583(7816), 360-360. doi: 10.1038/d41586-020-02086-x
Han, J., Pei, J., & Tong, H. (2022). Data mining: concepts and techniques. Morgan kaufmann.
Hassani, H., Huang, X., & Silva, E. (2019). Big data and climate change. Big Data and Cognitive Computing, 3(1), 12. doi: 10.3390/bdcc3010012
Höchtl, J., Parycek, P., & Schöllhammer, R. (2016). Big data in the policy cycle: Policy decision making in the digital era. Journal of Organizational Computing and Electronic Commerce, 26(1-2), 147-169. doi: 10.1080/10919392.2015.1125187
Hornby, P. and Perera, H.S.R. (2002) A development framework for promoting evidence‐based policy action: drawing on experiences in Sri Lanka. The International Journal of Health Planning and Management, 17(2), 165‐183. doi: 10.1002/hpm.660
Hossin, M. A., Du, J., Mu, L., & Asante, I. O. (2023). Big Data-Driven Public Policy Decisions: Transformation Toward Smart Governance. Sage Open, 13(4), 21582440231215123. doi: 10.1177/21582440231215123
Huang, M. H., & Rust, R. T. (2018). Artificial intelligence in service. Journal of service research, 21(2), 155-172. doi: 10.1177/1094670517752459
Hughes, W. R. (2009). A statistical framework for strategic decision making with AHP: Probability assessment and Bayesian revision. Omega, 37(2), 463-470. doi: 10.1016/j.omega.2007.07.002
Ibarra-Vazquez, G., Ramírez-Montoya, M. S., Buenestado-Fernández, M., & Olague, G. (2023). Predicting open education competency level: A machine learning approach. Heliyon, 9(11). doi: 10.1016/j.heliyon.2023.e20597
Igwama, G. T., Olaboye, J. A., Maha, C. C., Ajegbile, M. D., & Abdul, S. (2024). Big data analytics for epidemic forecasting: Policy Frameworks and technical approaches. International Journal of Applied Research in Social Sciences, 6(7), 1449-1460. doi: 10.51594/ijarss.v6i7.1334
Kenett, R. S., & Shmueli, G. (2016). Information quality: The potential of data and analytics to generate knowledge. John Wiley & Sons. doi: 10.1002/9781118890622
Kim, J., & Hastak, M. (2018). Social network analysis: Characteristics of online social networks after a disaster. International journal of information management, 38(1), 86-96. doi: 10.1016/j.ijinfomgt.2017.08.003
Kingdon, J. W. (1996) Young, E., & Quinn, L. (2002). Writing effective public policy papers. Open Society Institute, Budapest.
Kitchin, R. (2014). The data revolution: Big data, open data, data infrastructures and their consequences. Sage. doi: 10.4135/9781473909472
Kou, G., Peng, Y., & Wang, G. (2014). Evaluation of clustering algorithms for financial risk analysis using MCDM methods. Information sciences, 275, 1-12. doi: 10.1016/j.ins.2014.02.137
Kraft, M. E., & Furlong, S. R. (2020). Public policy: Politics, analysis, and alternatives. CQ Press.
Lee, Jung Wan (2020). Big Data Strategies for Government, Society and Policymaking. Journal of Asian Finance Economics and Business, 7(7), 475-487. doi: 10.13106/jafeb.2020.vol7.no7.475
Lew, M. S. (Ed.). (2013). Principles of visual information retrieval. Springer Science & Business Media. doi: 10.1007/978-1-4471-3702-3
Li, S. T., & Shue, L. Y. (2004). Data mining to aid policy making in air pollution management. Expert Systems with Applications, 27(3), 331-340. doi: 10.1016/j.eswa.2004.05.015
Li, S. T., Shue, L. Y., & Lee, S. F. (2008). Business intelligence approach to supporting strategy-making of ISP service management. Expert Systems with Applications, 35(3), 739-754. doi: 10.1016/j.eswa.2007.07.049
Machairas, E., & Varouchakis, E. A. (2023). Cost–benefit analysis and risk assessment for mining activities in terms of circular economy and their environmental impact. Geosciences, 13(10), 318. doi: 10.3390/geosciences13100318
Martínez-Álvarez, F., Troncoso, A., Asencio-Cortés, G., & Riquelme, J. C. (2015). A survey on data mining techniques applied to electricity-related time series forecasting. Energies, 8(11), 13162-13193. doi: 10.3390/en81112361
Morariu, C., Morariu, O., Răileanu, S., & Borangiu, T. (2020). Machine learning for predictive scheduling and resource allocation in large scale manufacturing systems. Computers in Industry, 120, 103244. doi: 10.1016/j.compind.2020.103244
Namdarian, Leila. (2016). Evidence-based policymaking and the role of statistics and information in it. Journal of Information Processing and Management, 31 (3), 601- 629. .[In Persian].doi: 10.35050/JIPM010.2016.007 [link]
Ngai, E. W., Hu, Y., Wong, Y. H., Chen, Y., & Sun, X. (2011). The application of data mining techniques in financial fraud detection: A classification framework and an academic review of literature. Decision support systems, 50(3), 559-569. doi: 10.1016/j.dss.2010.08.006
Niu, H., & Silva, E. A. (2020). Crowdsourced data mining for urban activity: Review of data sources, applications, and methods. Journal of Urban Planning and Development, 146(2), 04020007. doi: 10.1061/(ASCE)UP.1943-5444.0000566
O’Dwyer, L. (2004). A critical review of evidence-based policymaking. https://www.ahuri.edu.au/research/final-reports/58 
 OECD. (2015). Data-driven innovation: Big data for growth and well-being. OECD Publishing.
OECD. (2020). Building Capacity for Evidence-Informed Policymaking: Lessons from Country Experiences. OECD Publishing.
Ordu, M., Demir, E., Tofallis, C., & Gunal, M. M. (2021). A novel healthcare resource allocation decision support tool: A forecasting-simulation-optimization approach. Journal of the operational research society, 72(3), 485-500. Doi: 10.1080/01605682.2019.1700186
Ozkaya, G., Timor, M., & Erdin, C. (2021). Science, technology and innovation policy indicators and comparisons of countries through a hybrid model of data mining and MCDM methods. Sustainability, 13(2), 694. doi: 10.3390/su13020694
Parkhurst, J. (2017). The Politics of Evidence: From Evidence-Based Policy to the Good Governance of Evidence. Routledge.
Perini, D. J., Batarseh, F. A., Tolman, A., Anuga, A., & Nguyen, M. (2023). Bringing dark data to light with AI for evidence-based policymaking. In AI Assurance (pp. 531-557). Academic Press. doi: 10.1016/B978-0-32-391919-7.00030-5
Plotnikova, V., Dumas, M., & Milani, F. (2020). Adaptations of data mining methodologies: a systematic literature review. PeerJ Computer Science, 6, e267. doi: 10.7717/peerj-cs.267
Provost, F., & Fawcett, T. (2013). Data science and its relationship to big data and data-driven decision making. Big data, 1(1), 51-59. doi: 10.1089/big.2013.1508
Roiger, R. J. (2017). Data mining: a tutorial-based primer. Chapman and Hall/CRC.
Sarker, I. H. (2021). Data science and analytics: an overview from data-driven smart computing, decision-making and applications perspective. SN Computer Science, 2(5), 377. Doi: 10.1007/s42979-021-00765-8
Savaglio, C., & Fortino, G. (2021). A simulation-driven methodology for IoT data mining based on edge computing. ACM Transactions on Internet Technology (TOIT), 21(2), 1-22. doi: 10.1145/3402444
Shahbaz, M., Nasir, M. A., Hille, E., & Mahalik, M. K. (2020). UK’s net-zero carbon emissions target: Investigating the potential role of economic growth, financial development, and R&D expenditures based on historical data (1870–2017). Technological Forecasting and Social Change, 161, 120255. doi: 0.1016/j.techfore.2020.120255
Shepherd, B. (2020). Quantifying Trade Law: New Perspectives on the Services Trade Restrictiveness Index. Trade L. & Dev., 12, 270.
Shi, D., Guan, J., Zurada, J., & Manikas, A. (2017). A data-mining approach to identification of risk factors in safety management systems. Journal of management information systems, 34(4), 1054-1081. doi:10.1080/07421222.2017.1394056
Simonofski, A., Fink, J., & Burnay, C. (2021). Supporting policymaking with social media and e-participation platforms data: A policy analytics framework. Government Information Quarterly, 38(3), 101590. doi: 10.1016/j.giq.2021.101590
Sousa, M. J., Jamil, G., Walter, C. E., Au‐Yong‐Oliveira, M., & Moreira, F. (2023). Big data analytics on patents for innovation public policies. Expert Systems, 40(1), e12673. doi: 10.1111/exsy.12673
Tan Wu, Z., Ji, D., Yu, K., Zeng, X., Wu, D., & Shidujaman, M. (2021). AI creativity and the human-AI co-creation model. In Human-Computer Interaction. Theory, Methods and Tools: Thematic Area, HCI 2021, Held as Part of the 23rd HCI International Conference, HCII 2021, Virtual Event, July 24–29, 2021, Proceedings, Part I 23 (pp. 171-190). Springer International Publishing. doi: 10.1007/978-3-030-78462-1_13
Tsoukias, A., Montibeller, G., Lucertini, G., & Belton, V. (2013). Policy analytics: an agenda for research and practice. EURO Journal on Decision Processes, 1(1-2), 115-134. doi: 10.1007/s40070-013-0008-3
Valle-Cruz, D., Criado, J. I., Sandoval-Almazán, R., & Ruvalcaba-Gomez, E. A. (2020). Assessing the public policy-cycle framework in the age of artificial intelligence: From agenda-setting to policy evaluation. Government Information Quarterly, 37(4), 101509. doi: 10.1016/j.giq.2020.101509
Viet, N. T., Kravets, A., & Duong Quoc Hoang, T. (2021). Data mining methods for analysis and forecast of an emerging technology trend: a systematic mapping study from Scopus papers. In Artificial Intelligence: 19th Russian Conference, RCAI 2021, Taganrog, Russia, October 11–16, 2021, Proceedings 19 (pp. 81-101). Springer International Publishing. doi: 10.1007/978-3-030-86855-0_7
Weng, S., Schwarz, G., Schwarz, S., & Hardy, B. (2021). A framework for government response to social media participation in public policy making: Evidence from China. International Journal of Public Administration, 44(16), 1424-1434. doi: 10.1080/01900692.2020.1852569
Wirjo, A., Calizo,S., Nino Vasquez, G., & San Andres, E. A.(2022). Artificial Intelligence in Economic Policymaking. APEC Policy Support Unit. Retrieved from https://www.apec.org/publications/2022/11/artificial-intelligence-in-economic-policymaking
Witten, I. H., Frank, E., Hall, M. A., Pal, C. J., & Data, M. (2005, June). Practical machine learning tools and techniques. In Data mining (Vol. 2, No. 4, pp. 403-413). Amsterdam, The Netherlands: Elsevier.
Yadav, V. (2022). Machine Learning for Predicting Healthcare Policy Outcomes: Utilizing Machine Learning to Forecast the Outcomes of Proposed Healthcare Policies on Population Health and Economic Indicators. Journal of Artificial Intelligence & Cloud Computing, 1(2), 2-10. doi:10.47363/JAICC/2022(1)E112
Young, E., & Quinn, L. (2002). Writing effective public policy papers. Open Society Institute, Budapest. https://www.nccmt.ca/registry/resource/pdf/94.pdf
Quarterly Journal of Governance Knowledge
Volume 2, Issue 4
Autumn 2024
Pages 4-29