Alternate Forms Reliability – Calculation and Examples

Alternate forms reliability measures the consistency of scores obtained from two different versions of a test that are designed to measure the same construct. Researchers normally administer both forms to the same participants and correlate the resulting scores. Strong evidence also requires checking whether the forms have comparable content, difficulty, means, variances and score differences.

Introduction

Researchers often need more than one version of an assessment. A university may prepare different versions of an examination to protect test security. A psychologist may need an alternative memory test for repeated assessments. A longitudinal study may require comparable measures at several time points without asking participants exactly the same questions.

Alternate forms reliability helps determine whether scores remain consistent when the particular questions, tasks or stimuli change.

This article explains:

• What alternate forms reliability means
• How it is calculated
• How alternate, equivalent and parallel forms differ
• Why a high correlation does not automatically make two forms interchangeable
• How to develop and validate two forms
• How to interpret and report the results
• How modern software and artificial intelligence can support the process

Key takeaways

• Alternate forms reliability evaluates score consistency across two different versions of an instrument.
• The same participants should usually complete both forms so their paired scores can be compared.
• Pearson’s correlation coefficient is the traditional estimate, but correlation alone does not prove that scores agree.
• Researchers should also compare content coverage, item properties, means, variances and score differences.
• There is no universally acceptable reliability coefficient; the required precision depends on the intended interpretation and consequences of the scores.
• AI can assist with creating candidate items, but it cannot replace expert review, piloting and psychometric validation.

What is alternate forms reliability?

Alternate forms reliability is the degree to which two different versions of a measurement instrument produce consistent scores for the same people.

The two forms are intended to measure the same construct, ability, knowledge domain or characteristic. They contain different items or tasks, but they should follow the same content specifications and produce comparable score interpretations.

For example, a researcher develops two 30-item tests of statistical reasoning:

• Form A contains one set of problems.
• Form B contains different problems assessing the same topics and cognitive skills.

The same students complete both forms. If students who perform strongly on Form A also tend to perform strongly on Form B, the two score sets will have a positive correlation. Under appropriate form-equivalence assumptions, that correlation can be interpreted as an alternate forms reliability coefficient.

Alternate forms reliability is also called:

• Alternative-form reliability
• Equivalent-forms reliability
• Comparable-forms reliability
• Parallel-forms reliability
• Interform reliability

These terms are often used as synonyms in introductory research texts. However, they can have different technical meanings.

Are alternate forms and parallel forms the same?

They are often treated as synonyms, but “parallel forms” can refer to a stricter statistical model.

An alternate form is generally another version intended to measure the same construct. A strictly parallel form is expected to satisfy stronger classical-test-theory conditions.

Under a classical parallel-test model, the forms have:

• The same true scores for each person
• Equal error variances
• Uncorrelated measurement errors
• Comparable observed-score means
• Comparable observed-score variances

Perfectly parallel forms are difficult to create in practice. Researchers often work with approximately equivalent, tau-equivalent or congeneric forms instead. For that reason, it is helpful to state the exact evidence used to establish comparability rather than relying only on the label “parallel.”

Why is alternate forms reliability important?

Alternate forms reliability addresses a practical question:

Would conclusions about a person remain reasonably consistent if a different sample of suitable test items had been used?

Every finite test samples only part of a larger content domain. A mathematics examination cannot include every possible algebra or geometry problem. A vocabulary test cannot contain every word a person might know. Scores may therefore depend partly on the particular items selected.

Alternate forms reliability evaluates this form of content-sampling consistency.

Common reasons for using alternate forms

Researchers and assessment developers use alternate forms when they need to:

1. Reduce memory effects from repeating identical items.
2. Reduce practice effects caused by familiarity with previous questions.
3. Prevent answer sharing or item exposure.
4. Administer pretests and posttests without repeating the same instrument.
5. Track change across several testing occasions.
6. Maintain test security in certification or admissions examinations.
7. Compare digital and paper versions of an assessment.
8. Replace outdated items while preserving score meaning.
9. Create accessible versions that remain psychometrically comparable.
10. Build large item banks for computer-based or adaptive testing.

What source of error does alternate forms reliability measure?

In classical test theory, an observed score is represented as:

[
X = T + E
]

where:

• (X) is the observed score,
• (T) is the true-score component, and
• (E) is measurement error.

Alternate forms reliability is sensitive to differences caused by the selection of items or tasks. When the forms are administered on separate occasions, it may also reflect:

• Temporary changes in concentration
• Fatigue
• Motivation
• Learning
• Practice
• Mood
• Environmental conditions
• Changes in the underlying construct

The administration schedule therefore affects what the coefficient represents.

If both forms are completed during one closely controlled session, the coefficient primarily reflects form and item-sampling differences, although fatigue and carryover remain possible.

If the forms are administered several weeks apart, the coefficient reflects both form differences and temporal instability.

How is alternate forms reliability calculated?

The traditional procedure is:

1. Develop two forms intended to measure the same construct.
2. Administer both forms to the same participants.
3. Calculate a total or scale score for each form.
4. Correlate Form A scores with Form B scores.

For approximately continuous scores with a linear relationship, Pearson’s product–moment correlation coefficient is commonly used.

Pearson correlation formula

[
r_{AB} =
\frac{\sum_{i=1}^{n}(A_i-\bar{A})(B_i-\bar{B})}
{\sqrt{\sum_{i=1}^{n}(A_i-\bar{A})^2
\sum_{i=1}^{n}(B_i-\bar{B})^2}}
]

where:

• (A_i) is participant (i)’s Form A score,
• (B_i) is participant (i)’s Form B score,
• (\bar{A}) is the mean of Form A,
• (\bar{B}) is the mean of Form B, and
• (n) is the number of participants with paired scores.

The coefficient ranges from (-1) to (+1).

• A coefficient close to (+1) indicates that participants retain a very similar rank order across forms.
• A coefficient near zero indicates little linear consistency.
• A negative coefficient indicates that higher scores on one form tend to be associated with lower scores on the other, which would normally signal serious form or scoring problems.

Worked example

Suppose 12 students complete two hypothetical versions of a research-methods test.

Student	Form A	Form B
1	42	43
2	47	45
3	51	53
4	55	54
5	58	60
6	62	61
7	66	67
8	70	71
9	74	72
10	79	81
11	83	82
12	87	88

The descriptive results are:

Statistic	Form A	Form B
Mean	64.50	64.75
Standard deviation	14.51	14.62

The Pearson correlation is:

[
r = .994
]

The approximate 95% confidence interval using Fisher’s transformation is:

[
95%,CI = [.979,\ .998]
]

The mean paired difference is:

[
\bar{B}-\bar{A}=0.25
]

These hypothetical data provide strong evidence of score consistency because:

• The correlation is extremely high.
• The means are almost identical.
• The standard deviations are similar.
• The average score difference is small.
• The confidence interval remains high.

A complete analysis would still examine the content blueprint, item difficulty, item discrimination, order effects and subgroup performance.

Why correlation alone is not enough

Pearson correlation measures association and rank-order consistency, not exact agreement.

Imagine that every student receives exactly five more points on Form B:

[
B=A+5
]

The correlation would be:

[
r=1.00
]

However, the forms would not produce equal scores. Form B would systematically yield scores five points higher.

This example shows why a high coefficient cannot, by itself, demonstrate that two forms are interchangeable.

What should be examined in addition to correlation?

Researchers should normally examine:

• Mean of each form
• Standard deviation of each form
• Score range and distribution
• Mean paired difference
• Distribution of paired differences
• Outliers
• Item difficulty
• Item discrimination
• Internal consistency of each form
• Confidence interval for the reliability estimate
• Classification agreement when scores produce categories
• Performance across relevant demographic or ability groups

Depending on the research purpose, additional analyses may include an intraclass correlation coefficient, concordance coefficient, Bland–Altman plot, paired equivalence test, structural-equation model or item response theory analysis.

These methods answer different questions and should not be selected mechanically.

Reliability, agreement and equivalence

The terms reliability, agreement and equivalence should be distinguished.

Reliability

Reliability concerns the consistency or dependability of scores. A correlation-based alternate forms coefficient primarily shows whether participants retain a similar relative ordering.

Agreement

Agreement concerns whether the actual scores are sufficiently close. Two forms can correlate strongly while differing systematically in level or spread.

Equivalence

Equivalence concerns whether observed differences are small enough to fall within a prespecified acceptable range. Researchers must define this range substantively before analysing the data.

Failing to reject a conventional null hypothesis of no difference is not the same as demonstrating equivalence. An equivalence procedure such as two one-sided tests can be used when an appropriate equivalence margin has been justified.

Test equating

Equating is a separate procedure used to place scores from different forms onto a common reporting scale.

Two forms may have high alternate forms reliability but still differ slightly in difficulty. A testing programme may use statistical equating to adjust for those differences. Reliability evidence does not remove the need for equating when scores from different operational forms must be directly comparable.

How should alternate forms reliability be interpreted?

There is no universal coefficient that makes a test reliable for every purpose.

Interpretation should consider:

• The consequences of decisions based on the score
• The stability of the construct
• The length of each form
• Score variability in the sample
• The quality of the item pool
• The interval between administrations
• The intended population
• Whether individual or group decisions are being made
• The width of the confidence interval
• Other evidence concerning validity and fairness

A coefficient that may be adequate for exploratory group-level research may be inadequate for a high-stakes decision about an individual.

Avoid mechanical cutoffs

Rules such as “anything above .70 is acceptable” or “anything above .80 is strong” may be convenient summaries, but they should not replace contextual judgement.

Researchers should ask:

1. How precise do the scores need to be?
2. What would happen if a person received a meaningfully different result on the other form?
3. Does the lower confidence limit remain acceptable?
4. Are the forms equally accurate across the score range?
5. Do the forms produce consistent classifications?
6. Are the findings replicated in the intended population?

Why sample variability matters

A reliability coefficient is affected by between-person variability.

A heterogeneous sample with a wide range of ability may produce a high correlation because participants are easy to rank. A homogeneous sample with a narrow score range may produce a lower correlation even when the absolute score differences are small.

Researchers should therefore report sample characteristics and score distributions. Reliability should not be treated as a permanent property of an instrument independent of population and context.

Assumptions and design requirements

1. The forms measure the same construct

Both forms must represent the same intended knowledge, ability, attitude or behaviour. A high correlation does not correct poor construct alignment.

2. Content coverage is comparable

Each form should follow the same test blueprint. Topics, learning objectives, cognitive processes and item formats should be represented in similar proportions.

3. Score scales are comparable

The forms should have the same scoring range and scoring rules. If one form is scored out of 40 and the other out of 60, scores must be placed on a defensible common scale before they are compared.

4. Difficulty is similar

One form should not contain consistently easier items. Compare total-score means as well as item-level difficulty.

5. Score variability is similar

Large differences in standard deviations can indicate that one form differentiates participants more strongly than the other.

6. Administration conditions are controlled

Instructions, time limits, testing environment, equipment, scoring and accessibility arrangements should be comparable.

7. Paired observations are correctly matched

Each participant’s Form A score must be paired with that same participant’s Form B score. Misaligned rows can seriously distort the coefficient.

8. The relationship is appropriate for the selected coefficient

Pearson’s correlation assumes a reasonably linear relationship and is sensitive to influential outliers. A scatterplot should be examined before interpretation.

9. Order effects are controlled

If everyone completes Form A first, an apparent difference may be caused by practice, fatigue or learning rather than the forms themselves.

A counterbalanced design is preferable:

• Half of participants complete A then B.
• Half complete B then A.

Random assignment to order strengthens the design.

10. The sample represents the intended population

Reliability estimated among advanced students may not apply to beginners. Evidence from one language, age group, country or educational level may not generalise automatically to another.

How much time should pass between administrations?

There is no universally correct interval. The interval should match the source of error the study is intended to evaluate.

Same session

Advantages:

• The underlying construct is unlikely to change.
• Attrition is minimised.
• Testing conditions can be controlled closely.

Risks:

• Fatigue
• Carryover between similar items
• Practice
• Reduced motivation during the second form

Short delay

A delay of several days may reduce immediate recall while keeping genuine change relatively limited.

Risks include:

• Learning between sessions
• Changes in motivation or health
• Different environmental conditions

Longer delay

A longer interval may be appropriate when the instrument will be used repeatedly over time, but the resulting coefficient includes more temporal change.

The chosen interval should be justified by:

• The stability of the construct
• The intended real-world use
• The risk of recall or practice
• Participant burden
• Previous evidence for the instrument
• The research design

The interval and administration order should always be reported.

How to develop and test alternate forms

Step 1: Define the intended score interpretation

State:

• What the test measures
• Who will take it
• How scores will be used
• Whether decisions concern ranks, absolute scores or classifications
• The level of precision required

Step 2: Create a detailed test blueprint

The blueprint should specify:

• Content domains
• Learning objectives
• Cognitive levels
• Number of items
• Item formats
• Time limits
• Scoring rules
• Target difficulty distribution
• Accessibility requirements

Each form should satisfy the same blueprint.

Step 3: Develop a sufficiently large item pool

Create more items than are needed for the two final forms. Items can then be selected or matched according to content and psychometric properties.

Avoid making one form from the strongest items and the other from the leftovers.

Step 4: Conduct expert review

Subject-matter and measurement specialists should examine:

• Construct relevance
• Content coverage
• Cognitive demand
• Clarity
• Cultural appropriateness
• Accessibility
• Potential bias
• Similarity between paired items
• Accuracy of scoring keys and rubrics

Step 5: Pilot the items

Administer candidate items to people resembling the intended test population.

Examine:

• Item difficulty
• Item discrimination
• Distractor functioning
• Missing responses
• Completion time
• Ceiling and floor effects
• Internal consistency
• Participant feedback

Step 6: Assemble the forms

Match forms on:

• Blueprint coverage
• Number of items
• Difficulty
• discrimination
• Item format
• reading load
• time requirements
• scoring range

Classical test theory, item response theory or automated test-assembly procedures may be used, depending on available data and expertise.

Step 7: Use an appropriate administration design

The most direct design administers both forms to the same sample.

Where possible:

1. Randomly assign participants to AB or BA order.
2. Use the same administration conditions.
3. Record the time interval.
4. Prevent access to answers between sessions.
5. Document accommodations and deviations.

Step 8: Inspect the data

Before calculating reliability:

• Verify score ranges.
• Check missing values.
• Confirm participant matching.
• Produce histograms and scatterplots.
• Investigate outliers.
• Compare means and standard deviations.
• Examine score differences by order.

Step 9: Calculate the coefficient and confidence interval

Calculate Pearson’s (r) when the assumptions and score type make it suitable. Report a confidence interval rather than only a point estimate.

A nonparametric association such as Spearman’s rank correlation may be useful for ordinal data or strongly non-normal relationships, but it changes the interpretation and still does not establish agreement.

Step 10: Evaluate form equivalence more broadly

Examine:

• Paired mean difference
• Relative and absolute score differences
• Equality or practical similarity of variances
• Item-level properties
• Internal consistency of each form
• Score agreement
• Equivalence within a justified margin
• Classification consistency
• Differential performance across groups

Step 11: Revise and cross-validate

Remove or replace poorly functioning items, assemble revised forms and test them in a new sample.

Reliability evidence from the same data used to optimise the forms may be overly favourable. Independent cross-validation provides stronger evidence.

Alternate forms reliability compared with other methods

Method	What changes?	Main question	Typical analysis
Alternate forms reliability	Test items or tasks	Are scores consistent across different versions?	Correlation plus form-equivalence evidence
Test–retest reliability	Time	Are scores stable when the same test is repeated?	Correlation or agreement across occasions
Split-half reliability	Item subset within one administration	Are different parts of one test internally consistent?	Half-test correlation with an appropriate correction
Internal consistency	Individual items	Do items intended to form a scale behave coherently?	Alpha, omega or a model-based coefficient
Inter-rater reliability	Rater	Do different raters produce consistent scores?	Kappa, ICC or another agreement coefficient
Classification consistency	Form or occasion	Are people placed in the same decision category?	Agreement indices or decision-consistency methods

Alternate forms versus test–retest reliability

Test–retest reliability normally uses the same form twice. Alternate forms reliability uses different items designed to measure the same construct.

Test–retest reliability can be affected by memory for the original questions. Alternate forms reduce direct item recall but introduce additional error because the content differs.

Alternate forms versus split-half reliability

Split-half reliability divides one test into two parts and is principally an internal-consistency method. The halves are shorter than the full test, so the raw half-to-half correlation is not normally interpreted as the reliability of the complete instrument.

Alternate forms are separate instruments intended to be usable independently. Each form should represent the full blueprint and produce a complete score.

Randomly dividing one test into two sets does not automatically create two operationally equivalent full-length forms.

Alternate forms reliability versus internal consistency

Internal consistency asks whether items within one form function together. Alternate forms reliability asks whether scores generalise across different item samples.

Two forms can each have high internal consistency but correlate poorly with one another if they emphasise different content. Conversely, two forms may correlate strongly even when each form contains heterogeneous subdomains.

Both types of evidence may therefore be needed.

Advantages of alternate forms reliability

Reduces direct memory effects

Participants encounter different items, making it harder to reproduce answers simply from memory.

Supports repeated measurement

Researchers can assess change over time without repeatedly exposing participants to identical material.

Evaluates item-sampling consistency

The procedure tests whether conclusions depend heavily on the particular questions selected.

Supports test security

Multiple forms reduce the usefulness of leaked questions and answer sharing.

Useful for pretest–posttest designs

Equivalent forms can reduce the threat that posttest improvement merely reflects familiarity with the pretest.

Supports large-scale testing

Certification, admissions and educational programmes often need several forms for different dates or locations.

Limitations of alternate forms reliability

Developing two forms is expensive

The process requires a larger item pool, expert review, pilot testing and statistical analysis.

Genuine equivalence is difficult

Small differences in wording, content, difficulty or response format can alter what an item measures.

Participants must often complete twice as much testing

Long assessments can create fatigue, attrition and reduced motivation.

Order effects may distort the comparison

A fixed order can confound form differences with practice or fatigue.

Time adds another source of error

When administrations are separated, changes in participants may reduce the coefficient even when the forms are well constructed.

Correlation may conceal systematic bias

The forms may rank participants similarly while producing meaningfully different absolute scores.

Reliability is population-dependent

An estimate from one sample may not apply to another population or score range.

Item security can restrict piloting

High-stakes programmes may be reluctant to expose operational items during field testing.

Common mistakes

Mistake 1: Calling any two tests alternate forms

The forms must be designed to support the same score interpretation. Two different tests of vaguely related topics are not automatically alternate forms.

Mistake 2: Using different participant groups and simply correlating group scores

Ordinary alternate forms correlation requires paired scores from the same people. Designs involving different groups require other linking, equating or modelling procedures.

Mistake 3: Reporting only statistical significance

A significant correlation merely indicates evidence that the population correlation is not zero. Even a weak coefficient can become statistically significant in a large sample.

Report the coefficient, confidence interval and practical interpretation.

Mistake 4: Treating a high correlation as proof of identical difficulty

Correlation is unchanged when a constant is added to every score. Means and score differences must also be examined.

Mistake 5: Ignoring form order

Administering Form A first to everyone makes it impossible to separate form effects from order effects.

Mistake 6: Applying a universal cutoff

Reliability requirements should reflect the intended use and consequences of error.

Mistake 7: Failing to report the interval

A coefficient from forms completed back-to-back does not represent the same error sources as a coefficient obtained six weeks apart.

Mistake 8: Confusing alternate forms with test equating

Reliability evaluates consistency. Equating adjusts score scales. One does not substitute automatically for the other.

Mistake 9: Omitting item-level analysis

Similar total-score correlations can conceal weak items, different content emphasis or subgroup bias.

Mistake 10: Assuming an AI-generated form is automatically equivalent

Textual similarity is not psychometric equivalence. Generated items require the same content review, piloting, fairness checks and empirical analysis as human-written items.

Use in modern research

Educational assessment

Teachers and examination boards use alternate forms to:

• Reduce copying
• Replace exposed items
• Conduct make-up examinations
• Assess learning at multiple time points
• Compare cohorts
• Maintain secure item banks

For classroom tests, a teacher may use a test blueprint and matched item pairs. For high-stakes programmes, larger samples, formal equating and specialised psychometric procedures are normally required.

Psychological and neuropsychological assessment

Repeated cognitive or memory assessments are especially vulnerable to practice effects. Alternative word lists, images, stories or problem sets can reduce direct recall.

Researchers must still consider whether the alternative materials have comparable familiarity, emotional content, linguistic complexity and cultural relevance.

Clinical and intervention research

Alternate forms can be used before and after an intervention so that improvement is less likely to reflect memorisation of the original items.

However, the pretest and posttest must be sufficiently equivalent. Otherwise, an apparent treatment effect may actually reflect a difference in form difficulty.

Survey research

Researchers sometimes create alternative phrasings of questionnaire items. This can reveal whether responses depend heavily on wording.

Changing wording can also change meaning, social desirability or response processes. Expert review and cognitive interviewing may therefore be necessary before two questionnaires are treated as equivalent.

Computer-adaptive testing

Computer-adaptive systems draw different item sets for different examinees. Traditional fixed-form correlations may not fully represent the precision of adaptive scores.

Modern programmes often use item response theory, information functions, conditional standard errors and simulation studies to evaluate comparability and precision across adaptive administrations.

Criterion-referenced and pass/fail testing

When the main purpose is to classify examinees as pass/fail, competent/not competent or eligible/not eligible, classification consistency may be more important than rank-order correlation.

Researchers should report how often examinees receive the same classification across forms, particularly for scores near the cut point.

Digital tools for analysis

Excel

Place Form A scores in one column and matched Form B scores in another.

Use:

=CORREL(A2:A101,B2:B101)

Also calculate:

• Means
• Standard deviations
• Paired differences
• A scatterplot
• A histogram of differences

Excel is adequate for basic classroom analysis, but advanced confidence intervals, equivalence testing and item analysis may require additional procedures.

SPSS

A basic Pearson correlation can be obtained through:

Analyze → Correlate → Bivariate

Select the two form-score variables and request Pearson’s correlation.

Researchers should separately request descriptive statistics, paired comparisons, scatterplots and—where appropriate—an intraclass correlation or other agreement analysis.

R

A basic analysis can be conducted with:

cor.test(form_a, form_b, method = "pearson")

Useful supporting calculations include:

mean(form_a)
mean(form_b)

sd(form_a)
sd(form_b)

mean(form_b - form_a)
sd(form_b - form_a)

plot(form_a, form_b)
abline(0, 1)

Packages are also available for intraclass correlations, equivalence testing, Bland–Altman plots, item analysis, structural equation modelling and item response theory.

Python

Using SciPy:

from scipy import stats

result = stats.pearsonr(form_a, form_b)
print(result.statistic)
print(result.confidence_interval(confidence_level=0.95))

Using pandas:

data[["form_a", "form_b"]].corr(method="pearson")

Researchers should verify missing-data handling, participant alignment, assumptions and software defaults before reporting results.

Can artificial intelligence create alternate test forms?

Generative AI can assist with:

• Producing candidate item variations
• Suggesting distractors
• Rewriting reading passages
• Generating matched scenarios
• Classifying items by topic or cognitive level
• Checking duplication
• Producing preliminary test blueprints
• Supporting item-bank documentation

However, AI output should be treated as draft material.

Risks of AI-generated items

AI may produce:

• Factually incorrect content
• Multiple correct answers
• Implausible distractors
• Unintended clues
• Repetitive item structures
• Different reading difficulty
• Cultural or demographic bias
• Items that test superficial wording rather than the intended construct
• Content resembling protected or previously published assessment items

Human-in-the-loop workflow

A defensible process is:

1. Define the construct and blueprint without relying on AI.
2. Provide explicit item specifications.
3. Generate several candidate items.
4. Verify every factual statement and answer key.
5. Conduct independent expert review.
6. Check accessibility, bias and cultural appropriateness.
7. Pilot items with the intended population.
8. Estimate item difficulty and discrimination.
9. Compare the forms statistically.
10. Document where and how AI was used.

AI can reduce drafting time, but empirical reliability and validity evidence must come from appropriate data rather than from the model’s claim that the forms are equivalent.

How to report alternate forms reliability

A good report should include:

• Purpose of the two forms
• Intended population
• Sample size and characteristics
• Method used to develop the forms
• Evidence of content equivalence
• Administration order
• Time interval
• Descriptive statistics for each form
• Reliability coefficient
• Confidence interval
• Mean score difference
• Other agreement or equivalence evidence
• Internal consistency of each form, where relevant
• Item-level findings
• Limitations

APA-style reporting template

Two alternate forms of the [instrument name] were administered to [N] participants. Participants were randomly assigned to AB or BA administration order, with a [time interval] between forms. Form A produced a mean score of [M] and standard deviation of [SD], while Form B produced a mean of [M] and standard deviation of [SD]. Scores on the two forms were strongly correlated, (r(df)=[coefficient]), 95% CI ([lower, upper]). The mean paired difference was [difference]. Taken together with [item analysis/equivalence/agreement evidence], the findings provided [limited/moderate/strong] evidence that the forms produced consistent scores in this sample.

Example using the hypothetical data

Two alternate forms of a research-methods test were administered to 12 students. Mean scores were similar for Form A ((M=64.50, SD=14.51)) and Form B ((M=64.75, SD=14.62)). Scores were strongly correlated, (r(10)=.994), 95% CI ([.979,.998]). The mean paired difference was 0.25 points. These results provided strong preliminary evidence of alternate forms reliability, although validation in a larger independent sample would be required before operational use.

Conclusion

Alternate forms reliability evaluates whether different versions of an instrument produce consistent results for the same participants. Pearson’s correlation is the traditional coefficient, but a responsible evaluation does not stop there.

Researchers should establish common content specifications, control administration order and timing, compare item and score distributions, report uncertainty, investigate agreement and assess fairness. When decisions are high stakes, formal equating, classification-consistency analysis, item response theory or additional validation may be necessary.

Alternate forms are valuable precisely because they change the particular items. Their quality therefore depends on demonstrating that the change in content does not produce an unacceptable change in score meaning.

References

• American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). Standards for educational and psychological testing. American Educational Research Association.
• García-Pérez, M. A. (2013). Statistical criteria for parallel tests: A comparison of accuracy and power. Behavior Research Methods, 45, 999–1010. https://doi.org/10.3758/s13428-013-0328-z
• Kelley, K., & Pornprasertmanit, S. (2016). Confidence intervals for population reliability coefficients: Evaluation of methods, recommendations, and software for composite measures. Psychological Methods, 21(1), 69–92. https://doi.org/10.1037/a0040086
• Lee, A. J., Goodman, S. R., Bauer, M. E. B., Minehart, R. D., Banks, S., Chen, Y., Landau, R. L., & Chatterji, M. (2024). Validating parallel-forms tests for assessing anesthesia resident knowledge. Journal of Medical Education and Curricular Development, 11, 23821205241229778. https://doi.org/10.1177/23821205241229778
• Livingston, S. A. (2018). Test reliability—Basic concepts (Research Memorandum No. RM-18-01). Educational Testing Service.
• O, K.-M. (2024). A comparative study of AI-human-made and human-made test forms for a university TESOL theory course. Language Testing in Asia, 14, Article 19. https://doi.org/10.1186/s40468-024-00291-3
• Wyse, A. E. (2021). How days between tests impacts alternate forms reliability in computerized adaptive tests. Educational and Psychological Measurement, 81(4), 644–667. https://doi.org/10.1177/0013164420979656