Introduction
In this third course assignment, you are to assume the role of an industrial-organizational psychology practitioner on an external consulting team. The consulting team is tasked with conducting a 3-day item writing workshop for a group of subject matter experts (SMEs) who are writing items for a noncognitive assessment.
This assessment will be used to promote individual contributors to supervisory roles. On the first day of training, the SMEs drafted their assigned items based on the test blueprint, or test specifications. You understand that even when following item writing best practices, many items will not survive pilot testing, As a result, the SMEs need to create as many quality items as possible.
The first batch of items contains consistent flaws relating to failure to follow item writing best practices and fairness. You will develop an item development training handout for the second day of the training session that:
· Provides examples of problematic items that demonstrate the types of flaws you observed when reviewing the SMEs’ items.
· Specifies item flaws.
· Explains why the item is flawed.
· Revises the item to eliminate flaws.
Instructions
You are not expected to be an expert item writer for this assignment. Rather, this assignment is to allow you to apply your knowledge about item writing and defend your decisions. This will help you build your test development skills. At some point as an internal or external consultant, you will likely need to write items, review items, or contract with a vendor to purchase off-the-shelf or custom assessments.
Many test development principles apply across a variety of assessments, ranging from tests of knowledge, skills, or abilities to organizational assessments. Therefore, basic knowledge of test and assessment development principles will be useful in your career.
Use the Item Writer Training Handout Template provided in the Resources for this assignment to prepare your training handout focused on developing higher quality test items.
Note: The template provides a completed example to help you complete the rest of the template.
The items on the training handout are not the same items developed by the SMEs; rather, they reflect the types of errors that the SMEs made the day before. For each item on the template, provide the following information:
· Identify the flaw(s) with the item. (1–2 sentences per flaw in bold font; you may use a bulleted list format.)
· Explain why each identified flaw is a flaw to provide a teaching point to the SMEs. (1–2 sentences for each flaw—normal font, not bold.)
· Rewrite the item to improve its quality. Use the comment feature in Word or another form of tracking to highlight your change(s). Make sure your changes are easily distinguishable.
At the end of the handout, provide
at least three references to professional standards, best practices from reputable sources, or the scholarly literature on item/test development, test fairness, or test bias. These references may be from the course readings or from your independent research. Adhere to APA formatting and style guidelines for these references.
Additional Requirements
·
Writing style: Imagine you are a trainer providing tips to improve item writers’ skills. Write in a conversational style suitable for a business setting, rather than in a formal, academic style. Describe the flaws in a respectful manner so as not to offend participants, who may have made the mistakes you are calling attention to in the training materials. Feel free to be directive (you are the expert, after all), but remember your goal is to cultivate their item writing skills.
·
Font and font size: Times New Roman or Arial, 10 or 12 point.
·
References: Include a
minimum of three references to current authoritative and/or scholarly sources. Current means no more than five years old. Insert these references in the appropriate space on the Item Writing Training Handout Template rather than on a separate reference page.
·
APA format: Use APA style and formatting guidelines for all references. For an APA refresher, consult the Evidence and APA section of the Writing Center. See specific requirements for references under Instructions.
·
Rubric: Review this assignment’s rubric so that you understand how your instructor is going to evaluate your assignment.
Resources
·
Item Writer Training Handout Template [DOCX]
Download Item Writer Training Handout Template [DOCX].
·
Evidence and APA
.
