Category Archives: Writing contest

Writing Contest Winners: Round One

The first round of our Spring 2018 writing contest series had only three submissions, but they were strong posts which provided critical reviews to two texts about Evidence-Based Policy Making (EBPM). The upside of the low number of posts is that all three contestants won a prize.

Please join us in congratulating the winners!
Third place: Gulzhaina Mussagali

Third Place

Second place: Sagida Serikbayeva

Second Place.png

First place: Mariya Ippolitova

First Place.png

To learn how to submit a post for Round 2, click here.

Review of the article “The Relevance of Evidence-Based Policy Making (EBPM) in Public Management”

Abstract

This blog post is the review of the article “The Relevance of Evidence-Based Policy Making (EBPM) in Public Management” by Joseph Tham. The blog post analyses the article’s structure, information is given in it, and content as well as it gives examples from the policy-making process in the Kazakhstani context in order to clarify the relevance of arguments given in the article.

Этот блог является обзором статьи «Актуальность разработки политики на основе фактических данных (EBPM) в общественном управлении» написанный Джозефа Тамом. В блоге анализируется структура статьи и содержание, а также приводятся примеры из процесса разработки политики в казахстанском контексте, чтобы прояснить значимость аргументов, приведенных в статье.

Бұл блог Джозеф Тэмам жазған «Мемлекеттік басқаруда EBPM-тың өзектілігі»  мақаласына шолу болып табылады. Блогта мақаланың құрылымы мен мазмұны талқыланған, сондай-ақ мақалада келтірілген дәлелдердің маңыздылығын түсіндіру үшін Қазақстан контекстіндегі саясатты әзірлеу үдерісінен мысалдар келтірілген.

 

The article “The Relevance of Evidence-Based Policy Making (EBPM) in Public Management” by Joseph Tham is the review of the usage of Evidence-Based Policy Making (EBPM) in policy-making process in three different countries including the USA, the UK, and Australia.  In general, the article analyzes the effectiveness of using EBPM, the implication for public management as well as various views towards it in these three countries.

Overall, the article is well-structured and easy to follow. It contains several chapters devoted to the particular theme. On the one hand, it makes it clear and structured but on the other hand, it looks like the simple list of ideas without analyses and synthesis.  For instance, the usage of EBPM in three countries is written separately in three paragraphs and it will be better to add one more paragraph in which the author analyzes the situation in these three countries by comparing and contrasting evidence and comes to the consensus.

One more idea for improving the article is connected with the last paragraph which is devoted to the situation in Kazakhstan. In the Kazakhstani context, EBPM is a new phenomenon and is not used widely in the policy-making process. Therefore, it is clear that there is the shortage of evidence related to the EBPM and as the result; the author wrote a short review by using available information. Moreover, the author gives some recommendations for the implementation of EBPM in Kazakhstani context. As it was mentioned above, it will be better to give a more practical recommendation based on the experiences mentioned above three countries highlighting strong and weak sides in the usage of EBPM in the policymaking.

As a part of governmental institutions, the education system is considered to be one of the important sectors which formulate the frameworks for all levels of society. Therefore, the usage of EBPM in the education system is important in order to make educational policies more effective and successful. Unfortunately, many policies in the education system fail because of the several reasons. One of the main reasons is connected with the shortage of evidence and the luck of pre-preparation in the implementation process. For example, in the Kazakhstani context, one of the reforms initiated by the MoES is e-learning project is criticized widely.  E-learning is a large-scale state project included in the State Program of Development of Education of the Republic of Kazakhstan for 2011 – 2020 years. Its main goal of the project is to ensure equal access for all participants in the educational process to the best educational resources and technologies. Initially, the implementation of E-learning in the education system was divided into two parts. The first part of the programme contains 2011- 2015. However, according to the statistics of the national website of e-learning in Kazakhstan, the first part of the project did not reach intended indicators and there was a mismatch between target numbers and real situation. In addition, Kenzhebayev and Dalayeva (2014) state that some teacher of schools where e-learning system was introduced faced with challenges such as double filling the documentation: the electronic journals and the school journals. From this example, it can be seen that the mistakes made at the beginning of the projects had an impact on the whole implementation process. Therefore, it is important also use EBPM in the policy-making process in the education system. Moreover, the analyses of situation before the implementation of the policy can help avoid possible challenges or show if it works or not.

In conclusion, overall, the article is clear and informative since it gives important information about EBPM in several contexts. However, these are some points which need further development such as the comparison and synthesis of situations in the different context and giving more practical recommendations and coming to one conclusion after the review of all contexts. In general, the problem raised in the article can be applied and is relevant to the education system too since the policy-making process in the education system also needs EBPM in order to make it more effective.

 

 

 

References

MoES, (2012). Concept of e-learning in Kazakhstan’s education system: the first results, its introduction into the education system. Retrieved from: http://www.aio.kz/files/13pdf.pdf

Кenzhebayev, G., Baidildina, S., Dalayeva, T. (2012). Problems of development of e-Learning content in historical education on the republic of Kazakhstan. International Perspectives on Education. BCES Conference Books. Vol.10. Retrieved from http://bces.conference.tripod.com/sitebuildercontent/sitebuilderfiles/bces.conference.book.vol.10.20

 

Is Evidence Making a Policy?

Abstract: This post is a critical review of the article by Joseph Tham “The relevance of evidence-based policy making (EBPM) in public management” (2017) where I suggest some possible areas for improvement and include educational researchers’ position to EBPM approach.

Абстракт: Эта публикация представляет собой критический обзор статьи Джозефа Тама «Актуальность разработки политики на основе фактических данных в государственном управлении» (2017), где я предлагаю некоторые области для совершенствования и взгляд на данный подход со стороны исследователей образования.

Абстракт: Бұл басылым Джозеф Там жазған “Мемлекет басқармасындағы нақты мәліметтерге негізделген саясатты әзірлеудің өзектілігі” (2017) атты мақаланың сынап талдауы боп табылады. Осы жарияланымда мен жетілдіруді қажет ететін салаларды әрі берілген тәсілге білім саласының зерттеушілерінің көзқарасын ұсынамын.

“No one can doubt that basing your predictions about policy effectiveness on evidence is a good idea”

(Cartwright & Hardie, 2012, p.53)

The author reviews the ideas about evidence-based policy making in the US, the UK, and Australia; touches upon some challenges of its implementation and presents the implications for public management. At the end, he gives the quick overview of the situation in Kazakhstan. According to Tham (2017) despite the fact that EBPM has a substantial number of proponents among governors and policy makers in the number of countries, its slow effect and difficulty in identifying quality evidence undermine its credibility.

In the review, I argue that ideas lack analysis and connection between them. The author uses a variety of sources such as government reports, conference proceedings, books, presidential addresses, organization websites and others to support his ideas but the quotes fail to achieve their rhetorical purpose and convince the readers. Here is an excerpt from the text:

The U.S. Government Accountability Office (GAO) has clear guidelines on
performance measurement, and outcome evaluation. In a 2011 document titled
Performance Measurement, the GAO says that outcome evaluation:

“assesses the extent to which a program achieves its outcome-
oriented objectives. It focuses on outputs and outcomes
(including unintended effects) to judge program effectiveness
but may also assess program process to understand how
outcomes are produced.”

The Brookings Institution, a private think-tank has issued a recent report that
calls on ‘Strengthening Results-focused Government.’:

“It would help strengthen Americans’ confidence that their
government is able to effectively and efficiently tackle the
challenges we face as a nation.” (Page 1, Feldman Strengthening
Results-focused government)

The ideas are just introduced but not discussed and consequently, the coherence is lacking. One idea jumps into another. As a reader, I am wondering how is the existence of efficiency measurements linked to Americans’ trust in their authorities? More analysis of the quotes would lead me to the better understanding of the author’s goal. The conclusion also does not bring the whole paper together and only characterizes the US case leaving the discussion about other countries, challenges, and recommendations without any attention. I would like to see how the cases are interdependent and how Kazakhstani case is different from them.

Which of the implications should be taken into account in the Kazakhstani context? In this part, I would like to compare Tham’s ideas about EBPM in public management with the use of evidence in the educational field on the example of State Program for Education Development. Tham (2017) states that with the “support of the demand for evidence,  and support for the generation of research evidence, EBPM will be strengthened and widely used [in Kazakhstan]” (Tham, 2017, p. 12). However, no parallels were drawn with existing literature on the situation in the US, the UK, and Australia.

Moreover, when it comes to policy-making in the educational field, Bridges and Watts (2009) report about “the failure of policy-makers to take research findings properly into account” (p. 37). Ironically, they cite a number of research studies from Australia, UK, and the US which demonstrate that even solid evidence plays a minor role in changing practitioners or policy-makers decisions (Bridges & Watts, 2009). So even if the research evidence is generated, it is simply ignored because it is problematic to identify what should be counted relevant evidence in a particular context. According to Kettl (2017), in the UK case, the evidence is “facts, figures, ideas, analysis and research” (as cited in Tham, p.8).  This kind of data is already required from the policy makers in Kazakhstan when developing a policy. For example, the State Program of Educational Development in the Republic of Kazakhstan for 2011-2020 (MoES, 2010), which is the foundational program for all the education initiatives, has a special section “Analysis of the current situation” where all the facts and figures on the progress of a program or necessity for its implementation are described. No state program can be developed without “the demand for evidence” (Tham, 2017, p.12). Policy makers rely on think-tanks, experts, commissions, media but “academic research on social issues, including education, sits at the bottom of the list of resources” (Bridges & Watts, 2009, p.37).

Evidence-based policy making is a complicated process because even the essential part of it – the evidence is hardly generalizable. What worked in Western countries may not necessarily work in Kazakhstan, and even the kind of relevant evidence varies from department to department. In this post, I used the article on evidence-based policy to look at some aspects of policy making in education. However, it would be easier to follow the author’s way of thinking if he commented more on the way he interprets some quotations and made more conclusions for the reader on their connection.

References

Bridges, D., & Watts, M. (2009). Educational research and policy: Epistemological considerations. In D. Bridges, P. Smeyers & R. Smith (Eds.), Evidence-based education policy (36-57). United Kingdom: Wiley-Blackwell

Cartwright, N.,& Hardie, J. (2012). Evidence-based policy: A practical guide to doing it better. NY: Oxford University Press

Tham, J. (2017). The relevance of evidence-based policy making (EBPM) in public management. Unpublished manuscript, the Academy of Public Administration under the President of the Republic of Kazakhstan, Astana, Kazakhstan

Kettl, D. et al. 2017. No time to wait: building a public service for the 21 st century, National Academy of Public Administration.

MoES. (Ministry of Education and Science). (2010, December 7). State Program of Education Development in the Republic of Kazakhstan for 2011-2020. Retrieved from www.akorda.kz/upload/SPED.doc

 

 

Critical Review of the article “Evidence-Based Policy Making (EBPM) is wicked: a critical assessment from the periphery” by Joseph Tham

Abstract

This article is a journey of thought, led by the author, introducing us to the concept of Evidence-Based Policy Making (EBPM). It is written in a style, which is original and unusual for academic writing. However, after the journey you are left with more questions that when you started.

Эта статья является путешествием мысли на которое вы приглашены автором, описывающим концепт политических решений, основывающихся на фактах. Стиль повествования оригинален и далёк от обычного академического письма. Но, в конце данного путешествия у вас остаётся больше вопросов чем до него.

Бұл мақала сізді ойлану саяхатына шақырып отыр, сізді шақырған ой-пікірлерімен, фактілерге негізделген саяси шешімдер тұжырымдамасын сипаттайды. Повесть стилі түпнұсқа және әдеттегі академиялық жазудан алыс. Бірақ, осы сапардың соңында бұрыннан гөрі көп сұрақтар туыңдайды.

 

A journey of thought

First Steps

The article is written as a description of the process through which the author went, and the same style would be appropriate for a critical review of the said article. As a starting point, imagine being exposed to a catchy title of an article, which instantly makes you want to read it. You start reading and discover that it is easy to read, almost conversational, yet gets a point across. You start to understand what the EBPM concept means. The “imagine it’s you” approach helps you to start a journey of “wickedness”. You enjoy that there are questions in the introduction, because they grab your attention and make you think about what lies ahead.

However, you may think that the introduction is lacking a pitch, which will make it more interesting for you, the actual reason behind the “why do I care” question. A description of possible positive outcomes of applying EBPM in Kazakhstan may have been helpful in relating to this article.

The Walking Tour

The main body starts with setting the context for the journey, the need for a proposal for the implementation of EBPM. You think that using a theoretical example to set the scene is a good way to introduce a topic to a reader who is not familiar with it. You like the style chosen, there are interesting metaphors used throughout the text, such as the “lamppost” (Tham, 2017, p. 7) which may illuminate knowledge. However, here you pause to think about the target audience of this article. If it is aimed at the policy makers and civil servants, it may be too informal, if it is aimed at the laymen – what is the point of imagining being an expert in the field?

You think about the mentioned theory-practice divide and relate the importance of this topic to policymaking and implementation of said policies in education. This is exemplified in the case of inclusive education, where the policies and evidence are in place, however the practice and implementation are lacking (Mahlo, 2013). And for Kazakhstan, in education and other sectors of policymaking there are developments in creating the empirical basis, but there is still a need to create better tools for measuring and evaluating the quality of policies to reach EBPM (OECD, 2014).

As another step of the journey, you notice the organisational pattern of the article, which divides it into different sections. But, as you continue reading, you start getting confused and feel like you are jumping from topic to topic, because the sections seem disconnected and do not always link together seamlessly. More linking and connections between topics wold have create a more cohesive experience of the text.

You feel that some points and ideas, while you start to grasp them overall, may really benefit from additional examples and explanation, such as the whole “wicked problems” (Tham, 2017, p. 5) concept, prominent in the name of the article, but brought up as a topic only by the 5th page.  Even systems approach cited as best for solving wicked problems may need to be based on scientific evidence, as the two concepts often go hand in hand: “evidence-based policy also aims to clarify the interrelationship between different risk factors and different types of measures. This brings us to the systems approach” (Filtness, 2016, p. 13). Another example is that author claims that it is “difficult for the proponents to recognize the role of politics?”, but gives no example of this difficulty, an example of which may have brought you closer to understanding of the ideas in the article.

The abundance of questions throughout the article makes you want to answer all of them, even though they may be rhetoric questions, yet the aim of the article does not seem to be a dialogue between the author and the readers. This creates the idea that creating several blog posts on this topic would allow for a more back-and-forth format and create platform for further discussion, which may have created a further purpose for the reader.

You Got Us Where You Needed, What Next?

You find it ironic, that he conflicting dilemma described in this work is that evidence based policy making needs more evidence to prove that it is worthwhile.

In the end of the journey, even though you may disagree with the conclusions, you sit there, realising that now you know more than you did before reading this work, making it a worthwhile contribution of your time. But there is a thought nagging at the back of your brain: “What next?”. Overall, I was left with the same feeling as after finishing watching the first season of Westworld, or if I were to put it in the words of the author: “there are so many questions, with few answers” (p. 7, 2017, Tham).

 

References

Filtness, A. J. (2016). The application of systems approach for road safety policy making Deliverable 8.1 of the H2020 project SafetyCube. Loughborough. Retrieved from https://dspace.lboro.ac.uk/dspace-jspui/bitstream/2134/23723/1/Del_8.1_Final_281016.pdf

Mahlo, D. (2013). Theory and Practice Divide in the Implementation of the Inclusive Education Policy: Reflections through Freire and Bronfenbrenner’s Lenses. Mediterranean Journal of Social Sciences MCSER Publishing, 4(13), 163–170. https://doi.org/10.5901/mjss.2013.v4n13p163

OECD. (2014). OECD Public Governance Reviews Kazakhstan: Review of the Central Administration. OECD Publishing. Retrieved from https://books.google.kz/books?id=uYXkBQAAQBAJ&dq=Evidence+Based+Policy+Making+in+Kazakhstan&hl=ru&source=gbs_navlinks_s

Tham, J. (2017). Evidence-Based Policy Making (EBPM) is wicked: a critical assessment from the periphery.