full transcript

From the Ted Talk by James A. Smith: The method that can "prove" almost anything

Unscramble the Blue Letters

So if a p-value doesn’t answer the research question, why does the scientific conumitmy use it? Well, because even though a p-value doesn’t directly state the probability that the results are due to random canhce, it usually gives a ptrety reliable indication. At least, it does when used correctly. And that’s where many researchers, and even whole fields, have run into trouble. Most real studies are more complex than the tea exneieprmt. Scientists can test their research question in multiple ways, and some of these tests might produce a statistically sigciinfnat result, while others don’t. It might seem like a good idea to test every possibility. But it’s not, because with each ainotdidal test, the chance of a flsae positive increases. snrhaiecg for a low p-value, and then presenting only that analysis, is often called p-hacking. It’s like twrhoing darts until you hit a bullseye and then saying you only threw the dart that hit the bull’s eye.

Open Cloze

So if a p-value doesn’t answer the research question, why does the scientific _________ use it? Well, because even though a p-value doesn’t directly state the probability that the results are due to random ______, it usually gives a ______ reliable indication. At least, it does when used correctly. And that’s where many researchers, and even whole fields, have run into trouble. Most real studies are more complex than the tea __________. Scientists can test their research question in multiple ways, and some of these tests might produce a statistically ___________ result, while others don’t. It might seem like a good idea to test every possibility. But it’s not, because with each __________ test, the chance of a _____ positive increases. _________ for a low p-value, and then presenting only that analysis, is often called p-hacking. It’s like ________ darts until you hit a bullseye and then saying you only threw the dart that hit the bull’s eye.

Solution

  1. searching
  2. significant
  3. throwing
  4. community
  5. chance
  6. experiment
  7. pretty
  8. false
  9. additional

Original Text

So if a p-value doesn’t answer the research question, why does the scientific community use it? Well, because even though a p-value doesn’t directly state the probability that the results are due to random chance, it usually gives a pretty reliable indication. At least, it does when used correctly. And that’s where many researchers, and even whole fields, have run into trouble. Most real studies are more complex than the tea experiment. Scientists can test their research question in multiple ways, and some of these tests might produce a statistically significant result, while others don’t. It might seem like a good idea to test every possibility. But it’s not, because with each additional test, the chance of a false positive increases. Searching for a low p-value, and then presenting only that analysis, is often called p-hacking. It’s like throwing darts until you hit a bullseye and then saying you only threw the dart that hit the bull’s eye.

Frequently Occurring Word Combinations

ngrams of length 2

collocation frequency
null hypothesis 3
research question 2

Important Words

  1. additional
  2. analysis
  3. answer
  4. bullseye
  5. called
  6. chance
  7. community
  8. complex
  9. correctly
  10. dart
  11. darts
  12. due
  13. experiment
  14. eye
  15. false
  16. fields
  17. good
  18. hit
  19. idea
  20. increases
  21. indication
  22. multiple
  23. positive
  24. possibility
  25. presenting
  26. pretty
  27. probability
  28. produce
  29. question
  30. random
  31. real
  32. reliable
  33. research
  34. researchers
  35. result
  36. results
  37. run
  38. scientific
  39. scientists
  40. searching
  41. significant
  42. state
  43. statistically
  44. studies
  45. tea
  46. test
  47. tests
  48. threw
  49. throwing
  50. trouble
  51. ways