Applied Research on English Language

Detecting Halo Effects across Rubric Criteria in L2 Writing Assessment: A Many-facet Rasch Analysis

Hakimeh Ayoobiyan; Alireza Ahmadi

Volume 12, Issue 1 , January 2023, , Pages 159-176

https://doi.org/10.22108/are.2022.132503.1848

Abstract

This study applied multi-faceted Rasch measurement to investigate the halo effect in the performance-based assessment of writing across rubric criteria. Five raters who had received ... Read More

Factors Affecting Untrained Raters’ Cognition in Rating Oral Proficiency of English Language Learners: Insights from Iran

Nasimeh Nouhi Jadesi; Alireza Ahmadi; Seyyed Ayatollah Razmjoo

Volume 11, Issue 2 , April 2022, , Pages 1-24

https://doi.org/10.22108/are.2021.129400.1744

Abstract

Score decision-making is largely an undocumented process in performance assessment. To conduct a more in-depth cognitive study in scoring, one must ask if these underlying processes ... Read More

Cognitive and Psychological Characteristics and EFL Learners’ Test Performance: A Validity Study

Mohammad Ahmadi Safa; Rahime Karbakhsh ravari

Volume 11, Issue 1 , January 2022, , Pages 39-62

https://doi.org/10.22108/are.2021.130599.1778

Abstract

Language learners' test performance is affected by individual attributes that are not part of test takers’ language ability (Bachman, 1990). The identification of such factors ... Read More

The Status Quo of Language Assessment Literacy among Syrian EFL Teachers

Duaa Sulaiman; S. Susan Marandi; Leila Tajik

Volume 10, Issue 2 , April 2021, , Pages 33-60

https://doi.org/10.22108/are.2021.126136.1657

Abstract

With the increasing importance of Language Assessment Literacy in recent years, identifying the assessment literacy components in different contexts becomes essential to ensure that ... Read More

The Relationship between Graded and Tested Achievement: Do Gender and Proficiency Level Make a Difference?

Majid Nowruzi; Majid Amerian; Hooshang Yazdani; Alimohammad Mohammadi

Volume 9, Issue 4 , November 2020, , Pages 477-502

https://doi.org/10.22108/are.2020.120578.1516

Abstract

Grades represent one of the most common sources of evidence of student achievement in classrooms, though their relationship with test scores has remained understudied, particularly ... Read More

Cloze-Elide Test as an Alternative Test Method: Linking Personality Types to Test Method Performance

Rajab Esfandiari

Volume 8, Issue 1 , January 2019, , Pages 1-26

https://doi.org/10.22108/are.2018.113350.1373

Abstract

More innovative test methods may measure language learners’ test performance more accurately, contributing to much fairer decisions. This study examined Iranian language learners’ ... Read More

Operationalization of Formative Assessment in Writing: An Intuitive ‎Approach to the Development of an Instrument

Elaheh Tavakoli; Seyed Mohammad Reza Amirian; Tony Burner; Mohammad Davoudi; Saeed Ghaniabadi

Volume 7, Issue 3 , July 2018, , Pages 319-344

https://doi.org/10.22108/are.2018.112373.1340

Abstract

The current study aimed to develop a Formative Assessment of Writing (FAoW) instrument through operationalizing Black and Wiliam’s (2009) Formative Assessment (FA) and Hattie ... Read More

Iranian EFL Teachers' Language Assessment Literacy (LAL) under an ‎Assessing Lens

Mitra Janatifar; S. Susan Marandi

Volume 7, Issue 3 , July 2018, , Pages 361-382

https://doi.org/10.22108/are.2018.108818.1221

Abstract

Despite being trained in pre-service teacher education programs, most EFL teachers are underprepared when faced with language assessment-related activities. Part of the problem emanates ... Read More

Washback Effect of TEFL MA Exam on Iranian Lecturers’ Classroom Activities

Azadeh Pakzad; Hadi Salehi

Volume 7, Issue 1 , January 2018, , Pages 43-66

https://doi.org/10.22108/are.2018.106580.1179

Abstract

Washback refers to the effect of testing on teaching and learning. The university entrance exam for Iranian MA candidates of Teaching English as a Foreign Language (hereinafter TEFL ... Read More

Score Generalizability of Writing Assessment: the Effect of Rater’s Gender

Rahil Sheibani; Alireza Ahmadi

Volume 6, Issue 4 , November 2017, , Pages 411-434

https://doi.org/10.22108/are.2017.106097.1170

Abstract

The score reliability of language performance tests has attracted increasing interest. Classical Test Theory cannot examine multiple sources of measurement error. Generalizability theory ... Read More

Advanced and upper-intermediate EFL learners’ reciprocity to mediation: A dynamic listening assessment

Mehri Izadi; Hooshang Khoshsima; Esmaeel Nourmohammadi; Nahid Yarahmadzehi

Volume 6, Issue 4 , November 2017, , Pages 523-542

https://doi.org/10.22108/are.2018.108533.1210

Abstract

The present study aimed to capture and represent the mediator-learner’s interaction in the development of listening proficiency and statistically compare this interaction between ... Read More

Native and Non-native Teachers’ Pragmatic Criteria for Rating Request Speech Act: The Case of American and Iranian EFL Teachers

Minoo Alemi; Neda Khanlarzadeh

Volume 6, Issue 1 , January 2017, , Pages 67-84

https://doi.org/10.22108/are.2017.21326

Abstract

Abstract: Over the last few decades, several aspects of pragmatic knowledge and its effects on teaching and learning a second language (L2) ... Read More Abstract: Over the last few decades, several aspects of pragmatic knowledge and its effects on teaching and learning a second language (L2) have been explored in many studies. However, among various studies, the area of interlanguage pragmatic (ILP) assessment is quite novel issue and many features of it have remained unnoticed. As ILP assessment has received more attention recently, the necessity of investigation on the EFL teachers‟ rating criteria for rating various speech acts has become important. In this respect, the present study aimed to investigate the native and non-native EFL teachers‟ rating scores and criteria regarding the speech act of request. To this end, 50 American ESL teachers and 50 Iranian EFL teachers participated to rate the EFL learners‟ responses to video-prompted Discourse Completion Tests (DCTs) regarding the speech act of request. Raters were supposed to rate the EFL learners‟ responses and mention their criteria for assessment. The results of the content analysis of raters‟ comments revealed nine criteria that they considered in their assessment. Moreover, the result of the t-test and chi-square analyses of raters‟ rating scores and criteria proved that there are significant differences between native and non-native EFL teachers‟ rating pattern. The results of this study also shed light on importance of sociopragmatic and pragmalinguistic features in native and non-native teachers‟ pragmatic rating, which can have several implications for L2 teachers, learners, and material developers. معیارهای معلمان زبان بومی و غیربومی در نمره دهی کنش کلامی درخواست : مورد معلمان انگلیسی زبان آمریکایی و ایرانی چکیده: طی چند دهه اخیر، جنبه های متفاوتی از دانش منظور شناختی و تاثیر آن بر آموزش و فراگیری زبان دوم در تحقیقات متعدد مورد بررسی قرار گرفته است. با این حال، در میان تحقیقات مختلف، ارزیابی منظور شناختی بینازبانی تقریبا یک موضوع جدید محسوب می شود و بسیاری از جنبه های آن مورد بررسی قرار نگرفته است. به دلیل اهمیت زیادی که اخیرا ارزیابی منظور شناختی بینازبانی پیدا کرده است، ضرورت بررسی معیارهای سنجش معلمان انگلیسیِ بومی و غیربومی در نمره دهی کنش های کلامی آشکار گشته است. در این زمینه، تحقیق حاضر با هدف بررسی معیارها و نمره های داده شده توسط معلمان انگلیسیِ بومی و غیربومی در ارزیابی کنش کلامی درخواست صورت پذیرفته است. بدین منظور، 50 معلم انگلیسی زبان آمریکایی و 50 معلم انگلیسی زبان ایرانی برای ارزیابی مجموعه موقعیت آزمون تکمیل گفتمان زبان آموزان ایرانی مشارکت کردند. معلمان می بایست پاسخ های داده شده توسط زبان آموزان را نمره داده و معیار هایی که برای ارزیابی در نظر گرفته بودند، ذکر می کردند. در نتیجه ی تحلیل محتوای نظرات معلمان،9 معیار که معلمان در ارزیابی زبان آموزان در نظر گرفته بودند مشخص شد. علاوه بر این، نتیجه ی آزمون تی و مربع خی نمرات داده شده و معیارهای ذکر شده معلمان نشان داد که تفاوت معناداری بین روش های نمره دهی معلمان بومی و غیر بومی وجود دارد. همچنین نتایج به دست آمده از این مطالعه اهمیت ویژگی هایمنظور-جامعه شناختی و زبان-منظورشناختی را در ارزیابی منظور شناختی معلمان بومی و غیربومی مشخص کرده می تواند کاربرد گوناگونی برای معلمان زبان دوم، دانش آموزان، و طراحان کتاب های درسی داشته باشد. کلید واژه ها: نمره دهی منظور شناختی بین زبانی، ملاک نمره دهی، در خواست، آزمون موقعیت تکمیل گفتمان

Factors Influencing Iranian Untrained EFL Raters' Rating Group Oral Discussion Tasks: A Mixed Methods Design

Nasimeh Nouhi Jadesi; Seyyed Ayatollah Razmjoo; Alireza Ahmadi

Volume 5, Issue 2 , July 2016, , Pages 191-210

https://doi.org/10.22108/are.2016.20426

Abstract

Using a mixed methods design, the present study attempted to identify the factors influencing Iranian untrained EFL raters in rating group oral discussion tasks. To fulfil this aim, ... Read More Using a mixed methods design, the present study attempted to identify the factors influencing Iranian untrained EFL raters in rating group oral discussion tasks. To fulfil this aim, 16 language learners of varying proficiency levels were selected and randomly assigned to groups of four and performed a group discussion task. Thirty two untrained raters were also selected based on their volunteer participations. They listened to the audio files of the group discussions and assigned a score of one to six to each language learners based on their own judgments. They also provided comments on each language learners’ performance pointing to why they assigned such scores. The researchers had an interview with the raters after the rating session as well. The quantitative phase investigated whether linguistic features of accuracy, fluency, complexity and amount of talk were attended to by the raters in terms of having any relationship to the scores the raters assigned. Speech rate as an index of fluency and amount of talk turned out to be significantly correlated with the scores. Of more importance was the qualitative phase with the aim of identifying other factors that may account for the scores. The comments provided by the raters on each score and the interviews were codified based on Content Analysis (CA) approach. It was found that the raters attend not only to the linguistic features in rating oral group discussions, but they are also sensitive to the interactional features like the roles the participants take in groups tasks and the overall interaction patterns of the groups. The findings of this study may shed light on group oral assessment in terms of training the raters rating group oral tests and developing rating scales specific for group oral assessment. Persian Abstract:پژوهش حاضر، با بهره گیری از روش تحقیق ترکیبی به بررسی عوامل مؤثر بر ارزیابان آموزش ندیده ایرانی در ارزیابی فعالیت بحث گروهی می پردازد. بدین منظور، 16 زبان آموز از سطوح مختلف انتخاب و به صورت تصادفی به گروه های 4نفره تقسیم شده و یک فعالیت بحث گروهی را انجام دادند. 32 ارزیاب آموزش ندیده نیز بر اساس تمایل شخصی خود انتخاب شدند. ارزیابان، به بحث های گروهی ضبط شده گوش داده و بر اساس قضاوت خود نمره ای بین1 تا 6 به زبان آموزان اعطا کرده و همچنین نظرات خود را درباره ی مواردی که بر نمره دهی آنها مؤثر بود ارائه کردند. محقققین همچنین با ارزیابان مصاحبه ای در مورد نحوه نمره دهی آنها انجام دادند. بخش کمّی به بررسی ویژگی های زبانی دقت، فصاحت، پیچیدگی، و میزان نمونه های کلامی تولید شده توسط زبان آموزان و همبستگی آنها با نمره کسب شده توسط هر زبان آموز پرداخت. میزان گفتار، بعنوان شاخصی از فصاحت و میزان تکلم، با نمرات داده شده رابطه معناداری داشت. بخش کیفی مطالعه ی حاضر از اهمیت بیشتری برخوردار است چراکه در این بخش به کمک کدگذاری تحلیل محتوای نظرات ارزیابان، به تعیین سایر عواملی که بر ارزیاب در نمره دهی بحث گروهی مؤثر بودند پرداخته شد. نتایج حاصل حاکی از آن است که ارزیابان نه تنها تحت تأثیر عوامل زبانی کلام هستند بلکه به ویژگی های تعاملی نظیر نقش افراد در بحث گروهی، میزان مشارکت در بحث گروهی و الگوی کلی تعامل نیز در نمره دهی توجه دارند. نتایج این پژوهش می تواند در آموزش ارزیابان در فعالیت های ارزیابی شفاهی گروهی و همچنین در تولید چک لیست های ارزیابی مخصوص ارزیابی شفاهی گروهی مفید باشد.کلید واژه: ارزیابی شفاهی گروهی، ویژگی های زبانی، تعامل، روش تحقیق ترکیبی، تحلیل محتوا

Oral Presentation vs. Free Discussion: Iranian Intermediate EFL Learners’ Speaking Proficiency and Perception

Elaheh Sotoudehnama; Maryam Hashamdar

Volume 5, Issue 2 , July 2016, , Pages 211-236

https://doi.org/10.22108/are.2016.20427

Abstract

Speaking is a significant skill that many foreign language learners are trying to master. In this study, the effectiveness of two different speaking activities, i.e. oral presentation ... Read More Speaking is a significant skill that many foreign language learners are trying to master. In this study, the effectiveness of two different speaking activities, i.e. oral presentation vs. free discussion, was investigated from quantitative and qualitative points of view. To achieve this purpose, 44 intermediate learners from a language institute in Tehran participated in the study. Half of the participants experienced oral presentation while the other half (22 in two other classes) experienced free discussion for eight sessions. The two activities were exchanged between the two groups after the end of the quantitative phase of the study for eight more sessions. The results of an independent samples t-test performed on the scores ofthe speaking section of a sample Preliminary English Test (PET, 2012 ) after the first phase of the study (the first 8 sessions) indicated that the learners who experienced oral presentation significantly outperformed the learners who experienced free discussion though both activities proved to be significantly useful. Furthermore, the results of the interview with 10 participants from each activity through purposeful sampling, after the second phase of the study (the second 8 sessions) through thematic analysis indicated that both free discussion and oral presentation activities had some merits and demerits. Based on the results it can be advisable to include the two activities as complementary. چکیده: مهارت گفتاری مهارتی مهم به شمار میرود که زبانآموزان تلاش بر فراگیری آن دارند. پژوهش حاضر به بررسی کمی وکیفی کارآمدی دو نوع فعالیت گفتاریِ ارائه شفاهی و بحث آزاد در مهارت گفتاری زبانآموزان پرداخته است. به این منظور چهل و چهار نفر از زبانآموزان سطح متوسط در یکی از موسسات زبان تهران مورد مطالعه قرارگرفتند. در طول هشت جلسه، نیمی از زبانآموزان ارائه شفاهی داشتند در حالی که نیمی دیگر (22 نفر در دو کلاس دیگر) به بحث آزاد پرداختند. در پایان تحقیق کمی هر گروه برای هشت جلسه دیگر فعالیت دیگر را تجربه کرد. نتایج آزمون تی مستقل از بخش مهارت گفتاری نمونه آزمون PET(2012) پس از مرحله اول تحقیق (8 جلسه اول) حاکی از آن بود که زبان آموزانی که ارائه شفاهی داشتند در مقایسه با گروهی که به بحث آزاد پرداخته بودند، در مهارت گفتاری به میزان معنا داری پیشی گرفتند. به علاوه نتایج مصاحبه ی حاصل از تحلیل تم ها پس از مرحله دوم تحقیق (8 جلسه دوم) با 10 نفر از افراد هرفعالیت که به طور هدفمند انتخاب شدند ، حاکی از آن بود که هردو فعالیت دارای مزایا و معایبی هستند. بر اساس نتایج پیشنهاد می شود هر دو فعالیت به صورت مکمل با هم مورد استفاده قرار گیرند. کلمات کلیدی: بحث آزاد، ارائه شفاهی ،نظر زبان آموزان، مهارت گفتاری

The Washback Effects of High School Third Grade Exam on EFL Teachers’ Methodology, Evaluation and Attitude

Mohammad Ahmadi Safa; Farhad Jafari

Volume 5, Issue 1 , January 2016, , Pages 77-98

https://doi.org/10.22108/are.2016.20397

Abstract

Abstract: The widespread use of test scores for different educational and social decision making purposes has made the washback effect of tests a distinct educational phenomenon (Cheng, ... Read More

Testing a model of L2 communication among Iranian EFL learners: A path analysis framework

Nasser Fallah; Habibollah Mashhady

Volume 4, Issue 1 , February 2015, , Pages 59-78

https://doi.org/10.22108/are.2015.15494

Abstract

Using willingness to communicate (WTC) and socio-educational models as a framework, the present study aimed at examining ... Read More

A confirmatory study of Differential Item Functioning on EFL reading comprehension

Alireza Ahmadi; Touraj Jalili

Volume 3, Issue 2 , August 2014, , Pages 55-68

https://doi.org/10.22108/are.2014.15489

Abstract

The present study aimed at investigating DIF sources on an EFL reading comprehension test. Accordingly, ... Read More

The development of a Persian reading span test for the measure of L1 Persian EFL learners’ working memory capacity

Mohammadtaghi Shahnazari

Volume 2, Issue 2 , July 2013, , Pages 107-116

https://doi.org/10.22108/are.2013.15473

Abstract

This study describes working memory and developing and validating of an L1 Persian reading span test for the measurement of working memory of L1 Persian EFL ... Read More

Pinpointing the classifiers of English language writing ability: A discriminant function analysis approach

Reza Pishghadam; Mohammad Ali Shams

Volume 2, Issue 1 , February 2013, , Pages 45-60

https://doi.org/10.22108/are.2013.15463

Abstract

The major aim of this paper was to investigate the validity of language and intelligence factors ... Read More

The effects of Curriculum-Based Measurement on EFL learners' achievements in grammar and reading

Mansoor Tavakoli; Samira Atefi Boroujeni

Volume 1, Issue 2 , September 2012, , Pages 83-104

https://doi.org/10.22108/are.2012.15457

Abstract

The purpose of this study was to examine the effect of using Curriculum-Based Measurement (CBM) on the learners’ achievement in L2 grammar and reading in an EFL context and to ... Read More

Bringing Classroom-Based Assessment into the EFL classroom

Andrew Finch

Volume 1, Issue 1 , January 2012, , Pages 73-86

https://doi.org/10.22108/are.2012.15447

Abstract

This paper describes how English as a Foreign Language (EFL) teachers can bring reliable, valid, user-friendly ... Read More

Volume 12 (2023)

Volume 11 (2022)

Volume 10 (2021)

Volume 9 (2020)

Volume 8 (2019)

Volume 7 (2018)

Volume 6 (2017)

Volume 5 (2016)

Volume 4 (2015)

Volume 3 (2014)

Volume 2 (2013)

Volume 1 (2012)

Main Subjects = Testing

Abstract

Abstract

Abstract

Abstract

Abstract

Abstract

Abstract

Abstract

Abstract

Abstract

Abstract

Abstract

Abstract

Abstract

Abstract

Abstract

Abstract

Abstract

Abstract

Abstract

Abstract