Two comparison tables showing J E L code assignment by humans and four A I systems for economics articles.Top table (a): It has 13 rows and 8 columns, with rows 1 to 3 representing column headers. In row 1, columns 2 to 6 are merged for the header “Assigned J E L codes”, and columns 1, 7, and 9 are blank. In row 2, columns 1 and 2 are blank, and columns 3 to 8 are merged for the header “Based on abstract only”. In row 3, the column headers from left to right are “Article published in Journal of Economic Literature Volume 62 Issue 1 (March 2024)”, “Peer-reviewed Journal of Economic Literature publication”, “Chat G P T 3.5”, “Chat G P T 4”, “Copilot”, “Gemini”, “Average overlap with the final J E L codes”, and “Average difference in the number of J E L codes”. The data from the rows are as follows: Row 4: Column 1: Alesina, Alberto, and Marco Tabellini, “The Political Effects of Immigration: Culture or Economics?”; Column 2: D 72, H 23, J 11, J 15, K 37, R 23, Z 13; Column 3: D 72, J 61, J 15, Z 13, O 70, (42.86 percent, negative 2); Column 4: F 22, D 72, Z 13, H 53, R 23, (42.86 percent, negative 2); Column 5: D 72, J 15, J 61, (28.57 percent, negative 4); Column 6: Primary: F 22, J 15, Secondary: D 72, H 12, (28.57 percent, negative 3); Column 7: 35.72 percent; Column 8: negative 2.75. Row 5: Column 1: Zhuravskaya, Ekaterina, Sergei Guriev and Andrei Markevich. “New Russian Economic History”; Column 2: N 10, N 30, N 40, N 60, P 20, P 30, Z 13; Column 3: N 4, P 2, O 1, O 57 (0 percent, negative 3); Column 4: N 43, N 44, P 26, P 48 (0 percent, negative 3); Column 5: N 60, P 20, Z 13 (57.14 percent, negative 3); Column 6: Primary: N 13, O 10, Secondary: P 11, P 27 (0 percent, 0); Column 7: 14.29 percent; Column 8: negative 2.25. Row 6: Column 1: Capraro, Valerio, Joseph Y. Halpern, and Matjaž Perc. “From Outcome-Based to Language-Based Preferences”; Column 2: C 70, C 90, D 11, D 90, Z 13; Column 3: D 03, C 72, D 64, D 82, Z 13 (20 percent, negative 3); Column 4: C 72, D 91, Z 13 (20 percent, negative 2); Column 5: D 03, D 83, C 72 (0 percent, negative 2); Column 6: Primary: C 72, Secondary: D 03, A 12 (0 percent, negative 2); Column 7: 10.00 percent; Column 8: negative 1.5. Row 7: Column 1: Rohner, Dominic. “Mediation, Military, and Money: The Promises and Pitfalls of Outside Interventions to End Armed Conflicts”; Column 2: C 78, D 74, D 82, F 13, F 51, F 52; Column 3: F 51, H 56, D 74, O 19, P 48 (33.33 percent, negative 1); Column 4: F 51, F 52, O 19, D 74 (50 percent, negative 2); Column 5: F 51, H 56, O 19 (16.67 percent, negative 3); Column 6: Primary: C 72, Secondary: F 51, O 15 (16.67 percent, negative 3); Column 7: 29.17 percent; Column 8: negative 2.25. Row 8: Column 1: Goldin, Ian, Pantelis Koutroumpis, François Lafond, and Julian Winkler. “Why Is Productivity Slowing Down?”; Column 2: E 23, E 24, J 24, L 16, O 33, O 47; Column 3: O 47, E 22, F 14, D 24, O 30 (16.67 percent, negative 1); Column 4: O 47, E 22, F 43, O 33 (16.67 percent, negative 2); Column 5: E 23, E 24, J 24 (50 percent, negative 3); Column 6: Primary: O 33, Secondary: O 14, F 41 (16.67 percent, negating 3); Column 7: 16.75 percent; Column 8: negative 2.25. Row 9: Column 1: Hadavand, Aboozar, Daniel S. Hamermesh, and Wesley W. Wilson. “Publishing Economics: How Slow? Why so slow? Is Slow Productive? How to Fix Slow?”; Column 2: A 11, B 29; Column 3: A 14, D 13, G 14, O 30, M 21 (0 percent, positive 3); Column 4: A 11, A 14, O 33, J 44 (50 percent, positive 2); Column 5: D 83, L 82, M 37 (0 percent, positive 1); Column 6: Primary: A 11, Secondary: O 34, I 23 (50 percent, positive 1); Column 7: 25.00 percent; Column 8: 1.75. Row 10: Column 1: Woodford, Michael. “Beyond the Natural Rate: Stephen Marglin on the Instability of Market Economies”; Column 2: E 12, E 23, E 24, E 32, E 41, E 52; Column 3: B 22, E 12, E 32, E 52, E 62 (50 percent, negative 1); Column 4: E 12, E 32, E 61 (33.33 percent, negative 3); Column 5: B 22, E 12, E 32 (33.33 percent, negative 3); Column 6: Primary: B 22, Secondary: E 11, B 52 (0 percent, negative 3); Column 7: 29.17 percent; Column 8: negative 2.5. Row 11: Column 1: Average number of J E L codes; Column 2: 5.571; Column 3: 4.857; Column 4: 3.857; Column 5: 3.143; Column 6: 3.286; Column 7: Blank; Column 8: Blank. Row 12: Column 1: Average overlap with the final J E L codes; Column 2: Blank; Column 3: 23.27 percent; Column 4: 30.41 percent; Column 5: 26.53 percent; Column 6: 15.99 percent; Column 7: Blank; Column 8: Blank. Row 13: Column 1: Average difference in the number of J E L codes; Column 2: Blank; Column 3: negative 0.714; Column 4: negative 1.714; Column 5: negative 2.429; Column 6: negative 1.857; Column 7: Blank; Column 8: Blank. Bottom table (b): It has 13 rows and 5 columns, with rows 1 to 3 representing column headers. In row 1, columns 2 to 5 are merged for the header “Assigned J E L codes”, and column 1 is blank. In row 2, columns 1 to 3 are blank, and columns 4 and 5 are merged for the header “Based on whole article”. In row 3, the column headers from left to right are “Article published in Journal of Economic Literature Volume 62 Issue 1 (March 2024)”, “Working paper (presumably chosen by authors)”. “Peer-reviewed Journal of Economic Literature publication”, “Chat G P T 4, working paper”, and “Chat G P T 4, Final publication”. The data from the rows are as follows: Row 4: Column 1: Alesina, Alberto, and Marco Tabellini, “The Political Effects of Immigration: Culture or Economics?”; Column 2: D 72, J 11, J 15, J 61, Z 1 (42.86 percent, negative 2); Column 3: D 72, H 23, J 11, J 15, K 37, R 23, Z 13; Column 4: D 72, J 15, Z 13, F 22, J 61 (42.86 percent, negative 2); Column 5: D 72, H 23, J 11, J 15, K 37, R 23, Z 13 (100 percent, 0). Row 5: Column 1: Zhuravskaya, Ekaterina, Sergei Guriev and Andrei Markevich. “New Russian Economic History”; Column 2: N 00 (0 percent, negative 6); Column 3: N 10, N 30, N 40, N 60, P 20, P 30, Z 13; Column 4: N 43, N 44, P 26, P 36, Q 18 (0 percent, negative 2); Column 5: N 10, N 30, N 40, N 60, P 20, P 30, Z 13 (100 percent, 0). Row 6: Column 1: Capraro, Valerio, Joseph Y. Halpern, and Matjaž Perc. “From Outcome-Based to Language-Based Preferences”; Column 2: C 70, C 91, D 01, D 03, D 63 (20 percent, 0); Column 3: C 70, C 90, D 11, D 90, Z 13; Column 4: C 70, C 91, D 01, D 03, D 63 (20 percent, 0); Column 5: C 70, C 90, D 11, D 90, Z 13 (100 percent, 0). Row 7: Column 1: Rohner, Dominic. “Mediation, Military, and Money: The Promises and Pitfalls of Outside Interventions to End Armed Conflicts”; Column 2: D 74, H 56, N 40 (16.67 percent, negative 3); Column 3: C 78, D 74, D 82, F 13, F 51, F 52; Column 4: D 74, F 51, H 56, N 40, O 19 (33.33 percent, negative 1); Column 5: F 51, F 52, D 74, C 78, O 19 (66.67 percent, negative 1). Row 8: Column 1: Goldin, Ian, Pantelis Koutroumpis, François Lafond, and Julian Winkler. “Why Is Productivity Slowing Down?”; Column 2: O 40, E 66, D 24 (0 percent, negative 3); Column 3: E 23, E 24, J 24, L 16, O 33, O 47; Column 4: O 40, E 66, D 24 (0 percent, negative 3); Column 5: E 23, E 24, J 24, L 16, O 33, O 47 (100 percent, 0). Row 9: Column 1: Hadavand, Aboozar, Daniel S. Hamermesh, and Wesley W. Wilson. “Publishing Economics: How Slow? Why so slow? Is Slow Productive? How to Fix Slow?”; Column 2: A 11, B 31 (50 percent, 0); Column 3: A 11, B 29; Column 4: A 11, A 14, C 80, J 44, Y 90 (50 percent, positive 3); Column 5: A 11, A 14, B 29, C 89, J 01 (100 percent, positive 3). Row 10: Column 1: Woodford, Michael. “Beyond the Natural Rate: Stephen Marglin on the Instability of Market Economies”; Column 2: N A, N A; Column 3: E 12, E 23, E 24, E 32, E 41, E 52; Column 4: B 22, E 12, E 32, E 52, J 31 (50 percent, negative 1); Column 5: E 12, E 24, E 32, E 41, E 52 (83.33 percent, negative 1). Row 11: Column 1: Average number of J E L codes; Column 2: 3.167; Column 3: 5.571; Column 4: 4.717; Column 5: 5.714. Row 12: Column 1: Average overlap with the final J E L codes; Column 2: 21.59 percent; Column 3: Blank; Column 4: 28.03 percent; Column 5: 92.86 percent. Row 13: Column 1: Average difference in the number of J E L codes; Column 2: negative 2.333; Column 3: Blank; Column 4: negative 0.857; Column 5: 0.143. Below the tables, a note reads, “The listed articles appeared in the March 2024 issue (Vol. 62 No. 1) of the Journal of Economic Literature which was the current issue as of April 2024, available at: h t t p s colon: slash slash w w w dot a e a w e b dot o r g slash issues slash 775. The first number in parentheses under assigned J E L codes is the overlap with the final J E L codes and the second number is the difference in the number of assigned J E L codes compared to the final ones. Used A I systems: Chat G P T, h t t p s colon: slash slash chat dot open a i dot com slash; Copilot, copilot dot microsoft dot com; Gemini, h t t p s colon: slash slash gemini dot google dot com slash. J E L codes are presented in the sequence provided in papers and recommended by A I systems. Data (incl. information about selected working papers) and full responses of A I systems are available in the Appendix slash Supplementary Material”. |
