WEBVTT ffe1e91d-55f8-488b-ac33-2cf8fb578936-0 00:00:02.000 --> 00:00:08.019 A question we often hear is how do I know that the gap I'm ffe1e91d-55f8-488b-ac33-2cf8fb578936-1 00:00:08.019 --> 00:00:10.570 seeing isn't just random? 3a1d316c-5e0e-4923-8cf2-a585529d1b54-0 00:00:11.050 --> 00:00:14.050 Or how do I know if the gap I'm seeing is real? ba689e64-00d5-4179-9803-6710ac7e0ef1-0 00:00:14.850 --> 00:00:17.315 Certainly if you have two different numbers, then the ba689e64-00d5-4179-9803-6710ac7e0ef1-1 00:00:17.315 --> 00:00:19.690 difference between the numbers is technically real. b4e13364-37ee-4ad5-b37b-4f5fdd32a18b-0 00:00:20.490 --> 00:00:23.606 However, what we really want to know is, are the numbers b4e13364-37ee-4ad5-b37b-4f5fdd32a18b-1 00:00:23.606 --> 00:00:27.050 different enough that I should make changes in what I'm doing? 857c8087-3293-4afe-b786-71935b2dd306-0 00:00:27.450 --> 00:00:29.450 Are the numbers meaningfully different? e12f27d5-0790-43ee-b1e2-3386a4c0a2e2-0 00:00:30.010 --> 00:00:31.690 Let's start with a simple example. 8dfd5200-12e8-4d77-97ea-64e427832e71-0 00:00:32.440 --> 00:00:34.880 You have two groups of students in your class. 3238fa02-2c86-469d-80c6-3d52bf72cedf-0 00:00:35.360 --> 00:00:40.720 Group 1, taller students get DFW grades 10% of the time. d1888b14-cc1a-4c73-9fef-99663000e667-0 00:00:41.440 --> 00:00:47.200 Group 2 shorter students get DFW grades 11% of the time. 56cb4837-4ed4-46a9-aea4-41a30179411e-0 00:00:47.560 --> 00:00:50.600 Common sense says there's not much you should do in this case. 6f46b4a5-3fca-477e-96f4-66160b9df9b6-0 00:00:50.920 --> 00:00:54.840 But what if group 2 received non passing grades 15% of the time? 79990d07-2b10-4c53-8d1a-da162856bab1-0 00:00:56.480 --> 00:01:01.148 What about 20% of the time if there were 100 students in each 79990d07-2b10-4c53-8d1a-da162856bab1-1 00:01:01.148 --> 00:01:01.600 group? 80134bbf-2a55-4c9d-844a-10f600c11f1f-0 00:01:02.040 --> 00:01:05.758 Your intuition is probably saying this is starting to sound 80134bbf-2a55-4c9d-844a-10f600c11f1f-1 00:01:05.758 --> 00:01:06.440 meaningful. 4d7d0a5a-f2df-42f6-b337-32b3d1f2a7f6-0 00:01:07.080 --> 00:01:09.508 Maybe the taller students are sitting in front of the shorter 4d7d0a5a-f2df-42f6-b337-32b3d1f2a7f6-1 00:01:09.508 --> 00:01:10.840 students and blocking their view. 28804fc0-a822-419e-beec-7bccbf18035e-0 00:01:11.960 --> 00:01:15.008 Whatever the reason, there's probably something meaningful 28804fc0-a822-419e-beec-7bccbf18035e-1 00:01:15.008 --> 00:01:16.920 going on that is worth looking into. 82a97174-970d-4738-8dda-68f0b2442a1b-0 00:01:18.200 --> 00:01:21.023 On the other hand, suppose you're teaching a smaller class 82a97174-970d-4738-8dda-68f0b2442a1b-1 00:01:21.023 --> 00:01:23.320 where there are only 10 students in each group. fb78f192-ed66-450b-a0ef-577b0a9da68f-0 00:01:24.200 --> 00:01:27.698 If the rate for the two groups are still 10 and 20%, is that fb78f192-ed66-450b-a0ef-577b0a9da68f-1 00:01:27.698 --> 00:01:28.960 difference meaningful? 25e5ba06-399f-4dcc-b937-2069ef07e342-0 00:01:30.720 --> 00:01:34.242 The field of statistics has produced a number of methods 25e5ba06-399f-4dcc-b937-2069ef07e342-1 00:01:34.242 --> 00:01:38.011 that help us examine what our intuitions are telling us, but 25e5ba06-399f-4dcc-b937-2069ef07e342-2 00:01:38.011 --> 00:01:39.000 with more rigor. 3320279c-74d2-4b2a-b3b3-91d4710b7267-0 00:01:39.960 --> 00:01:43.395 In our case, we leverage effect size methods which look at the 3320279c-74d2-4b2a-b3b3-91d4710b7267-1 00:01:43.395 --> 00:01:46.667 number of students in each group, difference in non passing 3320279c-74d2-4b2a-b3b3-91d4710b7267-2 00:01:46.667 --> 00:01:49.884 rates between each group, and we calculate the odds that a 3320279c-74d2-4b2a-b3b3-91d4710b7267-3 00:01:49.884 --> 00:01:52.720 student in each group will get a non passing grade. 43d9a728-22ba-4fa9-8bca-e804adb34009-0 00:01:53.080 --> 00:01:55.703 The field of effect size statistics also recommends 43d9a728-22ba-4fa9-8bca-e804adb34009-1 00:01:55.703 --> 00:01:58.680 thresholds to situations that are likely to be meaningful. 30443fce-f13a-4692-9810-b031699e56a2-0 00:01:59.240 --> 00:02:03.181 In the end, we have the data, course grades, the patterns we 30443fce-f13a-4692-9810-b031699e56a2-1 00:02:03.181 --> 00:02:07.187 see, who gets more of each type of grade, and the statistical 30443fce-f13a-4692-9810-b031699e56a2-2 00:02:07.187 --> 00:02:10.160 likelihood that the difference is meaningful. 5e31a152-218e-41a6-8617-4f9175372039-0 00:02:10.560 --> 00:02:13.880 This is the information that we use to determine whether there 5e31a152-218e-41a6-8617-4f9175372039-1 00:02:13.880 --> 00:02:16.200 are detectable gaps in any of your courses. b2f9fceb-6471-4137-b59a-b71b02ca6ba5-0 00:02:16.560 --> 00:02:20.155 And this in turn leads to the notable findings that you see in b2f9fceb-6471-4137-b59a-b71b02ca6ba5-1 00:02:20.155 --> 00:02:20.840 your portal. a76b95cf-6d46-4a36-b1ee-b5ca35ddaa1f-0 00:02:21.360 --> 00:02:25.684 You have a deeper knowledge of what is going on in and outside a76b95cf-6d46-4a36-b1ee-b5ca35ddaa1f-1 00:02:25.684 --> 00:02:26.920 of your classroom. 3feec8de-3acb-47d0-b5b6-7788210401b2-0 00:02:27.480 --> 00:02:30.828 Collaborating with your peers and professional development 3feec8de-3acb-47d0-b5b6-7788210401b2-1 00:02:30.828 --> 00:02:34.574 colleagues, you can determine if there are changes you could make 3feec8de-3acb-47d0-b5b6-7788210401b2-2 00:02:34.574 --> 00:02:37.640 that might improve outcomes for your future students.