Is Evidence Making a Policy?

Abstract: This post is a critical review of the article by Joseph Tham “The relevance of evidence-based policy making (EBPM) in public management” (2017) where I suggest some possible areas for improvement and include educational researchers’ position to EBPM approach.

Абстракт: Эта публикация представляет собой критический обзор статьи Джозефа Тама «Актуальность разработки политики на основе фактических данных в государственном управлении» (2017), где я предлагаю некоторые области для совершенствования и взгляд на данный подход со стороны исследователей образования.

Абстракт: Бұл басылым Джозеф Там жазған “Мемлекет басқармасындағы нақты мәліметтерге негізделген саясатты әзірлеудің өзектілігі” (2017) атты мақаланың сынап талдауы боп табылады. Осы жарияланымда мен жетілдіруді қажет ететін салаларды әрі берілген тәсілге білім саласының зерттеушілерінің көзқарасын ұсынамын.

“No one can doubt that basing your predictions about policy effectiveness on evidence is a good idea”

(Cartwright & Hardie, 2012, p.53)

The author reviews the ideas about evidence-based policy making in the US, the UK, and Australia; touches upon some challenges of its implementation and presents the implications for public management. At the end, he gives the quick overview of the situation in Kazakhstan. According to Tham (2017) despite the fact that EBPM has a substantial number of proponents among governors and policy makers in the number of countries, its slow effect and difficulty in identifying quality evidence undermine its credibility.

In the review, I argue that ideas lack analysis and connection between them. The author uses a variety of sources such as government reports, conference proceedings, books, presidential addresses, organization websites and others to support his ideas but the quotes fail to achieve their rhetorical purpose and convince the readers. Here is an excerpt from the text:

The U.S. Government Accountability Office (GAO) has clear guidelines on
performance measurement, and outcome evaluation. In a 2011 document titled
Performance Measurement, the GAO says that outcome evaluation:

“assesses the extent to which a program achieves its outcome-
oriented objectives. It focuses on outputs and outcomes
(including unintended effects) to judge program effectiveness
but may also assess program process to understand how
outcomes are produced.”

The Brookings Institution, a private think-tank has issued a recent report that
calls on ‘Strengthening Results-focused Government.’:

“It would help strengthen Americans’ confidence that their
government is able to effectively and efficiently tackle the
challenges we face as a nation.” (Page 1, Feldman Strengthening
Results-focused government)

The ideas are just introduced but not discussed and consequently, the coherence is lacking. One idea jumps into another. As a reader, I am wondering how is the existence of efficiency measurements linked to Americans’ trust in their authorities? More analysis of the quotes would lead me to the better understanding of the author’s goal. The conclusion also does not bring the whole paper together and only characterizes the US case leaving the discussion about other countries, challenges, and recommendations without any attention. I would like to see how the cases are interdependent and how Kazakhstani case is different from them.

Which of the implications should be taken into account in the Kazakhstani context? In this part, I would like to compare Tham’s ideas about EBPM in public management with the use of evidence in the educational field on the example of State Program for Education Development. Tham (2017) states that with the “support of the demand for evidence,  and support for the generation of research evidence, EBPM will be strengthened and widely used [in Kazakhstan]” (Tham, 2017, p. 12). However, no parallels were drawn with existing literature on the situation in the US, the UK, and Australia.

Moreover, when it comes to policy-making in the educational field, Bridges and Watts (2009) report about “the failure of policy-makers to take research findings properly into account” (p. 37). Ironically, they cite a number of research studies from Australia, UK, and the US which demonstrate that even solid evidence plays a minor role in changing practitioners or policy-makers decisions (Bridges & Watts, 2009). So even if the research evidence is generated, it is simply ignored because it is problematic to identify what should be counted relevant evidence in a particular context. According to Kettl (2017), in the UK case, the evidence is “facts, figures, ideas, analysis and research” (as cited in Tham, p.8).  This kind of data is already required from the policy makers in Kazakhstan when developing a policy. For example, the State Program of Educational Development in the Republic of Kazakhstan for 2011-2020 (MoES, 2010), which is the foundational program for all the education initiatives, has a special section “Analysis of the current situation” where all the facts and figures on the progress of a program or necessity for its implementation are described. No state program can be developed without “the demand for evidence” (Tham, 2017, p.12). Policy makers rely on think-tanks, experts, commissions, media but “academic research on social issues, including education, sits at the bottom of the list of resources” (Bridges & Watts, 2009, p.37).

Evidence-based policy making is a complicated process because even the essential part of it – the evidence is hardly generalizable. What worked in Western countries may not necessarily work in Kazakhstan, and even the kind of relevant evidence varies from department to department. In this post, I used the article on evidence-based policy to look at some aspects of policy making in education. However, it would be easier to follow the author’s way of thinking if he commented more on the way he interprets some quotations and made more conclusions for the reader on their connection.

References

Bridges, D., & Watts, M. (2009). Educational research and policy: Epistemological considerations. In D. Bridges, P. Smeyers & R. Smith (Eds.), Evidence-based education policy (36-57). United Kingdom: Wiley-Blackwell

Cartwright, N.,& Hardie, J. (2012). Evidence-based policy: A practical guide to doing it better. NY: Oxford University Press

Tham, J. (2017). The relevance of evidence-based policy making (EBPM) in public management. Unpublished manuscript, the Academy of Public Administration under the President of the Republic of Kazakhstan, Astana, Kazakhstan

Kettl, D. et al. 2017. No time to wait: building a public service for the 21 st century, National Academy of Public Administration.

MoES. (Ministry of Education and Science). (2010, December 7). State Program of Education Development in the Republic of Kazakhstan for 2011-2020. Retrieved from www.akorda.kz/upload/SPED.doc

 

 

2 thoughts on “Is Evidence Making a Policy?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s