A guild was an association of craftsmen or merchants formed for the furtherance of their professional interests. Guilds began in ancient Rome and came to prominence in the Middle Ages. Operating at the city level, they were organized by profession, such as carpenter guilds, painter guilds, baker guilds, and many more. Only licensed guild members who adhered to set standards for quality were allowed to practice their craft in a particular city. Master craftsmen took on contractually obligated apprentices who would study and work under the master in order to learn the trade. Apprentices were bound to a master and would perform minor duties as assigned. They were provided with room and board but received no wages. After several years of learning the basic skills of the trade, the apprentice could be promoted to journeyman. Journeymen, no longer beginners, had demonstrated enough mid-level ability to be paid for their work. Slowly but steadily, they were given increased responsibility as they proved their competency in greater ways on larger and more important projects. Journeymen worked at this level until they were able to prove mastery of their craft to the satisfaction of a panel of guild masters. Only after they became master craftsmen were tradespeople given the right to operate their own shops. 

Extensive learning and demonstrated proficiency over time were the driving forces behind a steady stream of apprentices bonded to masters

Today the research industry has no such thing. Nothing prevents people from calling themselves researchers when their capabilities are only apprentice-level or worse. The result is the abandonment of professional standards. Who are you trusting your consumer and market research to – apprentices, journeymen, or master craftsmen? Leonardo de Vinci started as an apprentice painter and Benjamin Franklin was an apprentice printer. As a teenager, Henry Ford was an apprentice machinist. What these visionaries learned as apprentices, they applied on their years-long journeys to becoming master craftsmen.

The caliber of the consumer research most companies are conducting is in free fall

The ominous trend of demanding faster and cheaper research results in studies that take too many shortcuts and cut too many corners. Here are some things that CEOs don’t know about.

• In most companies, several layers of marketing executives stand between the primary information user (the CEO) and the information providers (the research department).
• Studies go up ladders and down chutes, processed by gatekeepers who don’t know how to tell good surveys from bad.
• More often than not, research has been made subordinate to marketing. This is fundamentally flawed because marketing’s goal is to influence people while the goal of research is to understand them.
• Most organizations’ research department heads are not scientists seeking to gain a deeper understanding of human behavior. They are marketers wanting to sell us something.
• Most marketing-led research knows little about information quality and cares even less. When you have the Count Chocula account, you proudly cite bought-and-paid-for research that shows it is “part of a balanced diet.”

This is happening not only inside organizations’ own research departments, but among research vendors, too  

Would you be surprised to learn that more than half of the people who provide consumer and market research services lack the education and training to conduct studies of any real value? That their real talents are in sales and client relationship management? In vendor companies, the highest-paid people with the greatest influence on your research almost always have little or no research backgrounds. The problem is not that there are no capable in-house researchers or contract research providers. There are excellent, good, fair, and poor ones, just like any other occupation. The problem is that research standards continue to deteriorate as master craftsmen are replaced by apprentices and amateurs. 

What skills do companies look for in Market Research Managers and Directors?

We took a closer look at hundreds of online job postings for Market Research Managers and Directors and here’s what we found.

More than three out of four job postings list a marketing degree as their educational requirement. Even at the MBA level, most marketing students take just one course in research. Graduate marketing degrees typically involve little or no training in Anthropology, Sociology, or Psychology. Behavioral science programs devote years to training students not only in the art and science of measuring human behavior, but also in acquiring deeper understanding of people’s attitudes, values, wants, needs, expectations, and behaviors. Master craftsmen know these human elements are critical to all real understanding, as without them, surveys are only going through the motions.

Most postings require industry-specific experience. We know why micro-specialization is favored – too many people wrongly believe researchers must know a great deal about a topic to be able to study it. Taken to the extreme, this would mean there is no point in studying anything we don’t already know, which pretty much puts the kibosh on our entire educational system. Our position is that companies have org charts full of product and category experts. What they really need but rarely have is someone expert in research theory, methods, instrument design, data collection, data processing, analytics, and reporting across all product and service categories. Demanding narrowly specialized experience is an indication of big-time inside-the-box thinking. How specialized does it get? Food company ads ask for not just experience in the food category, not just in the snack category, but in the highly specific salty snacks category, where the job will report to a salty snacks marketing manager who looks only at the tree and never sees the woods. The alternatives to salty snacks (what about crackers, cookies, nuts?) are not just other salty snacks, never mind the chips v. pretzels v. cheese snacks v. pork rinds debate.

Handicapped by the myopia of experts, bottled water industry clients told us the words distilled, purified, and ionized said “purity” to consumers. Every discussion group told us those words were nothing more than cheap tricks companies used to avoid saying “treated tap water.”  The clients were  shocked to hear these realities as they watched and listened from behind the mirror.

Overspecialization is for insiders, not for the rest of us

At the University of the West Indies, a student asked if I knew that soca music is combination of soul music and calypso music. Back in the States, I had learned Soul was Aretha and Calypso was Harry Belafonte, so the soca played in Trinidad was unrecognizable to me as either. Looking into it, I found that soca has nothing to do with the fusion of American Soul music and Calypso. Instead, Lord Shorty coined the term to describe the fusion of African Calypso and East Indian rhythms: “the (SO)ul of (CA)lypso.” 

What Caribbean islanders call calypso began long ago among plantation slaves

Wikipedia says sex, scandal, gossip, politics, and insulting other calypsonians were the order of the day in classic calypso, just as it is today in urban rhyming music. Over time, soca became highly specialized, fracturing into afro soca, bouyon soca, chutney soca, parang soca, power soca, ragga soca, and steelband soca. Each of these distinctions is a smaller piece of a smaller piece and is important only to an ever-smaller group of people.

Almost every job we found says the advertised position will report to a mid-level marketing executive

When research is located many layers away from the actual decision-makers, communication is altered as surely as you learned when you played the kid’s party game called Telephone or Chinese Whispers and saw how easily a simple sentence got mangled as it was passed from one person to the next. A companion study of the organizational charts of 200 global companies revealed nine out of ten market research departments reported to the marketing department. One consequence is that marketing becomes the gatekeeper, insuring that the research experts never see firsthand what the boss wants and the boss never sees what the research experts found. The farther away the research experts are from the boss, the less valuable the information, the more it costs, and the longer it takes.

The academic backgrounds, degree requirements, and reporting structures we observed indicate to us that when it comes to research, very few organizations understand the differences between apprentices, journeymen, and master craftsmen.

Who writes Research Manager and Research Director ads?

All kinds of people, including supervisors, subordinates, assistants, clerks, and others. When HR is involved, their primary concerns involve issues of legality, discrimination, and hiring quotas. Whoever writes them, most ads for Manager- and Director-level research jobs:

• Rely heavily on boilerplate that has little meaning.
• Use terms that facilitate resumé scanning.
• Are overwritten, trying to cover every eventuality.

After examining hundreds of ads for research supervisor positions, our panel of experts concluded only one was written by someone with an in-depth knowledge of research

In poker, a tell is a subtle clue readable by experts. In research, one tell is the heavy use of abbreviations and acronyms. When we saw VOC, NPS, SME, and the like in ads, we knew these organizations were missing the point. When we saw companies asking that candidates have experience with A.I. and Big Data, we knew they cared more about jargon and fads than about the underlying concepts. Companies should abandon their boilerplate ads and ask for research managers and directors who can:

• Manage the information gathering process.
• Run computer programs designed to calculate complex statistics on huge data sets.
• Analyze situations.
• Solve complex problems.

The lowering of hiring standards means research managers and directors are less likely than ever before to be journeymen or master craftsmen and more likely to be apprentices and even amateurs, the primary targets of DIY research companies.

Two claims shared by DIY research providers

Our product is easy to use. Most say their tools require absolutely no previous experience or knowledge. We believe the claim that people need no experience or training to create surveys like the ones Family Feud uses when they tell us they surveyed 100 members of their studio audience. Dig a little deeper on Free DIY Survey sites and you can sometimes find a section where they caution us about bias, phrasing, precision, and validity. This advice is helpful, but one can’t help but notice that they are telling us what to do without telling us how to do it or why it’s important. Noticeable by their omission are such fundamental issues as sequencing, flow, context, response categories, analytical plans, and the like. There are scores of things master craftsmen know to be crucial to good study design, and apprentices know nothing about them. Wikipedia lists more than one hundred types of bias. How many of them do you know?

Our product is fast. A typical claim is the ability to create and publish surveys quickly. This assertion that people can create surveys in minutes is true only at very simple levels. An excellent example is the email Hertz sends us after a rental, thanking us for our visit and asking us to click on either a green smiley face or a red frowny face. Calculating and reporting happy and unhappy responses can be done by a middle schooler.

Free DIY Survey companies deliver on their fast and easy to use promises by the heavy use of standardized formats and boilerplate questions. The one-size-fits-all approach fits only situations where the information goals are simple and there is little at risk.

Boilerplate

Boilerplate was originally a literal term that referred to iron rolled into large, flat plates for use in making boilers for the steam-powered manufacturing and shipbuilding industries. When these iron plates were all made to the same dimensions and specifications, boilers and ships could be built faster and cheaper. World Wide Words says the term was quickly adopted by news syndicates who provided material to the thousands of small-town papers around the country. Because the syndicated material was often third-rate filler stuff or thinly-disguised advertorials, boilerplate quickly came to mean hackneyed writing. Hackneyed has its own interesting origin, first meaning a horse rented for drudge work, then applied to the literary drudges (hacks) who cranked out that third-rate filler.

Today, boilerplate refers to any standardized piece that can be cut-and-pasted to save time and money

It is common in those online user agreements that are so numbingly long that we don’t read them. It is ready-made content, design, or format that is used without alteration, just like so many surveys, DIY in particular. Boilerplate design means the people we are surveying are required to shoehorn their thoughts into our format. Apprentice and journeymen survey writers make this kind of mistake over and over, never stopping to think about designing surveys that make the user experience pleasant and seamless.

This is the same backwards engineering that brought us all those annoying telephone menus and all those damnable sites that send us to endless FAQ pages that answer every question but the one we have. How many times have you filled out a survey that:

• Went into too much detail?
• Was too long and tiresome?
• Made you rate and rank things that aren’t important to you?
• Bored you so badly you answered mindlessly just to get it over with?
• Yelled at you go back and answer those questions you left blank?
• Annoyed you so badly that you quit?

Tail Wags Dog

Most surveys are designed backwards because researchers pull out the same stale boilerplate. All of us have seen two different ways online companies ask for our addresses. The old style follows the pattern established in 1963, when the US Postal Services introduced zip codes that were added to the bottom of the form. Handwritten and typed envelopes followed this format:

NAME _________________________

STREET ADDRESS _________________________

CITY _________________________

STATE _________________________

ZIP _________________________

Many of the online sites we see continue to use this half-century old boilerplate. Asking us to enter our city and state is fossilized nonsense because the city and state are already encoded in our five-digit zip codes. This is apprentice-level thinking. Only a few well-designed sites ask for our zip first, then autofill our city and state for us. This is the hand of the master craftsman.

Here are a few favorites from the Hall of Shame 

Most online sites ask us to enter the month our credit card expires. Some list the month by its name (August, September) and others list it by its number (8, 9). This apprentice-level design forces us to do their coding for them. The best method uses both (8 –  August, 9 – September). When we buy gasoline, the pump asks us to enter our five-digit zip code. When we do, nothing happens, because the boilerplate user interface insists we push ENTER, too. Bad design. How about when we have to go back and add dashes to our phone number to fit their boilerplate format? Once you start looking around, you will find there are far too many of these unfriendly boilerplate forms that make us do the work the computers should be doing.

Marketing and Market Research sound alike 

Most companies assume all marketers have market research skills because marketing and market research sound so much alike. Astrology and Astronomy sound a lot alike, too, but one is a pseudoscience, while the other is a real science, which is the systematic study of the structure and behavior of the physical and natural world through observation and experiment. Sounding alike is a very poor rationale to put the two together. In fact, there is one powerful reason why they should not – the goal of marketing is to influence people, while the goal of research is to understand them.

Amateurs and Apprentices have taken over consumer and market research

Did you know that to become a barber requires the successful completion of a two-year apprenticeship? Yet MBA programs say your research apprenticeship is satisfied by taking a single course that requires you attend a class only three hours a week for only one semester (for-profit companies sell apprenticeships that last only a few days or a few weeks). My graduate school professor said he was going to assume the class had no prior knowledge of statistics, so he would start this required course at the very beginning. I felt I had an advantage because I had completed two statistics courses as an undergraduate. In only 45 minutes he had blown by everything I had ever learned about statistics and there were 35 more hours to go. Yikes. If you’re a college graduate, you had to take three or four increasingly difficult semesters of a foreign language to meet graduation requirements. Did you become proficient, expert, fluent in that language? Probably not. How fluent is your team in the language of consumer research? What are the credentials of the people who manage the flow of information up and down between the research department and the C-suite?

Click here to scroll through the articles I’ve written every week for LetsTakeACloserLook.com since 2016, always showing there is more to most things than meets the eye. Scroll down to to the bottom of this page, enter your email address and get a new article in your inbox every Monday. There is no charge, I don’t share your address with anyone, I send you no e-mails, there are no ads on the site and I reply personally to everyone who sends me a note. How often do you find those things these days?