Table 2. Results Type of question...

Table 2.

Results

Type of question	Available marks	ChatGPT (%)	Claude (%)	CoPilot (%)	Grok (%)	Gemini (%)	Student average (%)	Student pass rate (%)
Journal entries	12	41	100	41	86	75	63	80
Calculations for financial statement preparation	21	33	55	0	60	38	52	66
Calculations for financial statement preparation with partial solution given	32	19	0	19	19	63	47	53
Theory application	25	42	42	26	50	60	52	60
Critique the given accounting treatment	18	6	0	6	14	17	65	81
Total	108	27	31	17	40	51	54^a
Deviation from the student average on a total basis		−27	−22	−37	−14	−3

Type of question	Available marks	ChatGPT (%)	Claude (%)	CoPilot (%)	Grok (%)	Gemini (%)	Student average (%)	Student pass rate (%)
Journal entries	12	41	100	41	86	75	63	80
Calculations for financial statement preparation	21	33	55	0	60	38	52	66
Calculations for financial statement preparation with partial solution given	32	19	0	19	19	63	47	53
Theory application	25	42	42	26	50	60	52	60
Critique the given accounting treatment	18	6	0	6	14	17	65	81
Total	108	27	31	17	40	51	54^a
Deviation from the student average on a total basis		−27	−22	−37	−14	−3

Note(s):

Italic figures represent which model and the student average scored best for each type of question. Underlined figures represent the type of question in which each model and the student average performed best. ^aA final mark of 50% or more is required to pass at the authors’ home institution

[ViewLarge]

Sharing Unavailable