The game show Let’s Make a Deal was first shown on American television in 1963. It has been overhauled several times since then, with new hosts, models, and sets. In the original version, Monty Hall chose contestants from screaming audience members who dressed in costumes to catch his eye – the more outrageously, the better. Contestants would join Monty and floor model Carol Merrill onstage and compete for prizes. After several nonsense games, a finalist was chosen to play the Big Game for the Big Prize. The curtain rose to show three doors on the stage.
Monty said hidden behind the doors are one extravagant grand prize and two worthless booby prizes (different prizes every show). After telling everyone what today’s prizes were, he asked the contestants to choose one of the doors. Before Carol opened that door, she opened one of the other two, revealing one of the two booby prizes. Monty always gave contestants another choice: you can stick with the door you’ve chosen, or you can have the other unopened door. Most people think the odds are 50/50 either way, but they’re wrong.
The Monty Hall Problem is easily solved by statisticians because they understand the underlying logic and math involved in calculating probabilities (casinos call them the odds). Abandoning your first choice and taking the other door pays off twice as often as sticking with your first choice. This is as much a bafflement to some people as how airplanes are able to fly.
Here are three key takeaways:
- Things are not always how they appear to be.
- When new information emerges, re-evaluate your decisions.
- You’re far better off knowing what’s behind the numbers than relying on your intuition.
Journalist Susan Adams wrote in Forbes about a company getting rich by selling test answers to students cheating their way through college. Chegg sells a $14.95 a month online, on-demand service that employs 77,000 experts with advanced degrees in math and science. 24/7 freelancers located mostly in India deliver answers to exam questions in real time. College students call it chegging. The 2020 pandemic, which drove most university courses online, has been a boon for Chegg’s business and it is now a $12 billion dollar company.
Adams says Chegg is cashing in on students’ worst instincts.
She believes that “Our arsenal of digital tools and global connectivity should be deployed to transform education for the better, but Chegg is using them to outsource cheating to India. That is a tragedy.”
When I first read about Chegg, I was struck by the similarities with Cliffs Notes, who started the book summary business in 1958. When assigned books to read and summarize, some university students wouldn’t bother. Instead, millions turned in the short outlines written by others as work they had done themselves. Millions of Cliffs Notes were sold to students who were satisfied to go through college as corner-cutters and cheaters. Sensitive to these criticisms, Cliffs Notes said their outlines were never intended to replace reading the assigned books, and “Students who use them in this way are denying themselves the very education that they are presumably giving their most vital years to achieve.”
The word is derived from the Latin plagiare, which means to kidnap or abduct. Four hundred years ago, its meaning was extended to include “the wrongful appropriation of another’s ideas and giving them forth as one’s own.” Plagiarism is intellectual theft and a clear violation of the code of ethics and behavior that most academic institutions have established to regulate the scholastic conduct of their members. The surveyed employers who want new hires to have analytical and numeracy skills also want new college graduates to have certain attributes. None reported a desire to hire people who openly and willingly cheat on exams. Across the board, employers want to hire people who have a strong sense of right and wrong. Study results say they believe only one in four new college graduates make ethical decisions.
Professors everywhere believe huge numbers of students are chegging.
Chegg officials disagree, saying only a small number of students have misused the service, which is intended to help with homework. The reality is that thousands of subscribers copy from Chegg’s database of 46 million exam answers and submit them as work they did honestly.
A real-world example of chegging.
North Carolina State University teacher Tyler Johnson said he caught 250 students who had used Chegg to cheat on the final exam of his Intro to Statistics course. I recently spoke with Tyler, who believes Chegg absolutely knows their main business is selling answers to cheaters who believe:
- Advancing their self-interests by violating rules is more important than being honest.
- Getting better grades is now achievable without having to be bothered with all that tiresome studying and learning.
- The ability to fake knowledge is better than actually knowing and understanding how things work.
These students who are openly and willingly cheating on exams are applying for jobs with your company.
A study of digital-era employers showed they want new hires who have the digital-world skills required to:
- Analyze complex situations.
- Solve difficult problems.
- Apply their knowledge to the real world.
Digital-world skills like these require high levels of numeracy. We all know literacy is the ability to read and write. Its cousin numeracy is the ability to use and interpret mathematical information to solve real-world problems. More than 60% of students claim they have the skills necessary to analyze situations, solve problems, and apply what they know to real-world situations. Employers say only 20% of them actually do. This huge disconnect is because employers’ perceptions are based upon their real-world experiences while students are just bragging.
The ability to think abstractly and critically is a hard-to-acquire skill.
Why are critical thinkers in high demand in the modern data-driven world? Because they have learned to:
- Be open-minded and thorough.
- Ask the right questions.
- Determine the credibility of sources.
- Judge the quality of any premise by the extent to which its reasons, assumptions, and evidence support the conclusion.
- Develop and defend a reasonable position regarding a belief or an action.
Data science is the engine that runs today’s business world.
Oracle defines data science as the overlapping areas of statistics, scientific methods, and analysis – all of which are used to extract meaning and insights from data. Machine learning, artificial intelligence, cloud storage, and really big data means businesses have a growing need for people who understand at least the basics of data science that are at the heart of numeracy. The Program for the International Assessment of Adult Competencies defines numeracy as “the ability to find, use, interpret, and communicate mathematical information and ideas so as to engage in and manage the mathematical demands of the workplace.” Numeracy includes:
- Working with numbers, measurements, and statistical concepts.
- Computing, interpreting, and communicating the results.
- Working with mathematical models to predict future needs.
The PIAAC measured adults’ abilities to perform numerical tasks.
They found only 12% of U.S. adults have the necessary training to think conceptually, a skill that requires the ability to:
- Understand complex and abstract mathematical information.
- Choose relevant problem-solving strategies and processes.
- Develop and work with mathematical models.
- Communicate well-reasoned explanations for conclusions.
Abstract thinking is the ability to interpret complex ideas and draw conclusions.
Abstract thinkers understand the possibilities, the significance, and the deeper meaning that goes beyond the numbers. Critical thinking skills are developed by reading books because reading requires focus and guides deeper thinking. Reading cultivates in us the knowledge, habits, and skills needed to make decisions, plan, and prioritize; abstract thinking skills are developed by learning statistics.
Statistics are hard.
So are many things of value, like learning how to think critically and analyze insightfully. Those with even a little statistical training know that much of what lies within large and complicated data sets is hidden in the nooks and crannies. Deep insights are the result of subtle and nuanced understanding of huge data sets, not just numbers on a page. And a great deal of what is inside the data is counterintuitive, just like The Monty Hall Problem.
Only 12% of U.S. adults understand complex and abstract mathematical information.
The result is that eight out of nine new college graduates applying for your company’s research, data, information, and knowledge management jobs don’t have the numeracy skills they need. Heck, half of them think the ability to work with numbers and statistics is unimportant. They don’t think they need to know anything more than how to push Zoltar’s button for the answer.
Two ways to stop students from online cheating.
Sarah Eaton, a University of Calgary education professor, studies ethics and integrity in higher education. She says colleges and universities have two ways to fight online cheating: online proctoring and authentic assessment. Online proctoring uses technology to track exam-takers’ eye movements, analyze their keystrokes, and block them from unauthorized websites. Students find avoiding detection is easy by taking the test on one device while looking up answers on another. Like everything involving the interweb, the cycle of hack, patch, hack, patch never ends.
Demonstrating you know something deeply is another matter.
Instead of supplying “answers,” authentic assessment requires students to apply what they have learned in new circumstances.
Grant Wiggins coined the term authentic assessment in 1989. He described it as an effective measure of intellectual ability “because it requires students to demonstrate deep understanding, higher-order thinking, and complex problem-solving through the performance of real-world tasks.” In an interview with edutopia, Wiggins said authentic assessment has become more buzzword than reality. Rebelling against typical fill-in-the-blank, multiple-choice, and short-answer exams, he wanted to measure the real work real people actually do. “After all,” he said, “in the workplace, they don’t give you a multiple-choice test to see if you’re doing your job.”
Authentic assessment requires students to demonstrate their ability to apply what they’ve learned to real-world situations.
And that’s one of the big things companies want from recent graduates. Wiggins, a consultant on school-reform issues, said this type of testing is avoided by most teachers because it is labor-intensive and time-consuming. “Students come to believe that learning is cramming. Teachers come to believe that tests are imposed nuisances composed of contrived questions irrelevant to their intent and success. Both believe getting the answers right matter more than habits of mind.”
Indiana University says authentic assessments give students a chance to demonstrate the use of sound judgment in determing what information is relevant and how it should be used.
How one teacher caught 250 cheggers.
Chegg customers take photos of exam questions and and post them to online forums. Tutors post the answers. All paying customers now have access to all questions and answers. The quality of answers varies as widely as you would expect from a staff of nearly 80,000 tutors.
When North Carolina State University reacted to the pandemic of 2020 by going to remote classes, Tyler Johnson simplified the three statistics courses he was teaching. He made exam questions easier than usual because of the pressures and stresses on students. You might think Johnson’s Intro to Statistics course is just about complicated math, but it isn’t. Students are asked questions that require clear explanations of experimental designs and relevant variables.
Tests are not about formulas and slide rules, but about thinking through problems.
Johnson said his exams are designed to measure statistical literacy. While grading the midterm exams of his 800-student class, his assistants noticed some peculiar patterns of language and syntax. Johnson says his team’s sharp eyes and well-tuned ears detected most of the exam answers were written by people who do not have English as their first language. As Johnson described it, “We pulled a thread and kept on pulling until the entire sweater unraveled.”
Their quick detective work pointed to Chegg as the source.
Johson bought a subscription, investigated some more, and found dozens of cheaters who had blatantly cut-and-pasted answers. The mid-term exam cheggers had given themselves away because they didn’t bother to check the answers they bought from Chegg. The typos and grammatical errors were what Johnson called the smoking gun. His team went to the university with their findings. Pressured by university officials, Chegg provided detailed black box data but no names, due to privacy regulations. And now Johnson, a statistics teacher, had what he needed.
The final exam.
Johnson warned his students not to use any homework help sites. He told them anyone who did would be treated as cheaters and dealt with accordingly. On the final exam, clever cheaters had corrected obvious grammatical and spelling errors. But they had failed to check Chegg content against class content. Answers provided by Chegg tutors used concepts, terms, and definitions never introduced in class. Johnson and his team watched in real time as 250 cheaters cut and pasted the exact same exam answers from forums devoted to answering his test questions. Johnson said the Chegg answers submitted by cheaters puts the lie to Chegg’s claim of academic expertise. Some of their online tutors are quite knowledgeable, but most are underpaid freelancers with limited knowledge. The quality of the answers provided by cheggers? “Most graded no better than a C,” he said, “because their work is truly mediocre.”
Tyler Johnson would make a great detective.
He had anticipated Chegg would give him no names, so he constructed the final exam in such a way that each had unique identifiers. These allowed his team to track and determine exactly which students posted which questions in forums. Johnson is the rare teacher that writes totally fresh exams every time. Unlike most teachers, he does not recycle old questions or use boilerplate questions and answers from the textbook. Johnson is a tech whiz, too. He believes most cheaters can be caught, but admits that few professors in other fields have the technical savvy needed to ferret out the cheaters. He believes that as more students get caught, they will learn safer ways to cheat. Cheating by paying for online answers is rampant and there is no end in sight to this game of Whac-a-Mole.
Were his cheggers students who can’t do the work?
“Quite the contrary,” he said. “My assistants and I talk with them one-on-one throughout the semester, and these conversations make it easy for us to see most students have the ability to do the work. The majority are capable, but don’t want to spend the time because it is faster to get the answer and move on.” He added that they don’t want to expend the effort, they want a max-out grade, and they think the reward outweighs the risk.
Some think students’ attitudes are a reflection of how they were raised.
Today’s college students were born around the turn of the century. The only world they know is one with instant access to anything. When the world is only a few clicks away, why bother to learn anything when we can look it up in seconds? When caught chegging, some students shrugged it off with the childhood excuse that “everybody else is doing it.”
Teachers across the country share chegging stories.
One professor at a prestigious unversity said he gave students a take-home exam with explicit instructions that they were not to use Chegg under any circumstances. After hand-grading the exams, he saw how the solutions turned in by students did not match what he had taught in class. He paid for a Chegg subscription and after doing the kind of matching Johnson did, found three out of four of his students had used Chegg to cheat.
Academic deans, offices of student conduct, and integrity commissions around the country are treating the chegging problem differently.
NC State fully supported Johnson’s investigation by pressuring Chegg on his behalf. “We got the answers we were looking for,” he said, “but Chegg gave them only grudgingly. They will not help identifying cheaters unless the full weight of the university is behind the demand for access to their records.” Johnson sums it up the situation this way: “For $14.95 a month, Chegg sells students the opportunity to damage their academic careers and their future.”
The United States Air Force Academy Honor Code.
After going to online learning in 2020, the Air Force Academy suspended 249 cadets for violating the Honor Code. Infractions were primarily for using unauthorized online websites to receive answers to exam questions in real time. Most of the students who admitted to cheating were placed on probation and remediation. The Academy says more than 90% of cadets who go through remediation do not commit another Honor Code infraction. General Richard Clark, Academy Superintendent, said the Honor Code serves as “a guide for cadets to live an honorable life and serves as a fundamental priority for developing leaders of character.”
Very interesting, you say, but how giving me a real-world application?
How about using this article to completely rethink how you hire high level A.I., Machine Learning, and Knowledge Management candidates? Instead of a typical interview, try using an authentic assessment. Ask them to explain to a non-statistician like yourself:
- What steps they take to determine the credibility of information sources.
- How they judge the merits of a proposition.
- Their keys to determining the value and quality of data.
And by all means, ask them to describe times when they analyzed complex situations, solved difficult problems, and applied that knowledge to the real world.
Those who read and understand this article can judge for themselves what kind of candidate they are speaking with.
True professionals know what they’re doing and why. When they break the rules it is only because they understand them so well and recognize when exceptions are called for. Others believe that even if they don’t know what they are doing, they will probably get away with it. The irony is that there are more bullshitters than true professionals running A.I. and Machine Learning departments, most of them thriving because they go undetected.
When we ask applicants to talk about things that are supposed to be near and dear to their hearts, they quickly sort themselves into one of two groups.The best candidates delight in being able to demonstrate how well they know what these things are and why they are important. Let your competitors hire the others.
Click here for a link to Jimmy Buffett’s song Door Number Three.
To read more articles like this, click here.
<My apologies to subscribers. This article was accidentally posted yesterday while it was still being edited, before images were added, and with notes to myself still in there – a very large oops!>