Blog FPO

Getting Smart About Research

This work is licensed under a Creative Commons Attribution-NonCommercial 2.5 License.

This guest blog post is by Polly Karpowicz, MBA, CAE, and Elizabeth Weaver Engel, M.A., CAE.

Association executives use research all the time. We run surveys and focus groups to better understand our members’ most important goals and most pressing problems so we can make good decisions about the programs, products, and services we offer to meet those needs. We sponsor industry studies to help our members and other audiences make good decisions about investing their resources wisely. We consult third-party (university, government, NGO) data sets and studies to understand the STEEP (social, technological, environmental, economic, and political) trends that influence the operating environments of the professions and industries we serve. 

The problem is that most of us lack formal training in research methods.

How can you be sure that the research you’re using or sponsoring is giving you the insight you need to make good decisions? How can you protect your association’s reputation as a trusted source of unbiased information for the profession or industry you serve?

You need to build your information literacy muscles.

That involves asking questions like:

  • How was the data the study uses collected? 
  • Who was included in the research? Who was excluded?
  • Who collected the data? Are those sources reputable? 
  • Does the study draw conclusions without data to support them? 

It also involves familiarizing yourself with the terminology of research (sampling methods, margin of error, confidence interview, statistical significance, data hygiene, different types of research design bias) and with your options for collecting and interpreting data and the pros and cons of each, for instance:

Quantitative research, which is designed to ask questions and collect responses numerically, with the goal of coming to the kinds of answers that can be described by levels of statistical significance and degrees of confidence. Surveys are the most common form of quantitative research, and they provide reassuringly specific answers. However, surveys are particularly susceptible to design flaws in either the questions, the answer options, or both that can bias the results.

Qualitative research, which is designed to gather data through conversation. Interviews and focus groups are the most common types of qualitative research. While qualitative results are necessarily impressionistic and anecdotal, they allow you to delve into subjects’ motivations, the why behind what they think, prefer, and do. However, interviewers walk a fine line between creating a high level of trust with interview subjects, which is necessary for success, and inappropriately influencing their responses which can bias results.

Does this all sound complicated? 

It’s important for associations to get this right, both so that association executives have the best possible chance of making good decisions about how to invest limited association resources to generate the best return for members, and because associations are viewed as trusted, unbiased sources of information for the members and other audiences we serve. It’s incumbent on us to provide quality research products so we remain worthy of that trust. 

You can learn more about these and other critical research concepts and read stories of associations doing great work in their research programs to serve their members and the professions and industries they serve in our new whitepaper, Caveat Emptor: Becoming a Responsible Consumer of Research, in which we tackle the sometimes thorny issue of what you need to know and do to be a savvy consumer and sponsor of research even if you DON’T have a formal background.

The monograph also includes:

  • An interview with Dr. Sharon E. Moss, co-editor (with Sarah C. Slater) of The Informed Association: A Practical Guide to Using Research for Results, on ethical practices in research.
  • An interview with Dr. Joyce E. A. Russell, The Helen and William O’Toole Dean at Villanova School of Business, on developing discernment in assessing research.
  • An interview with Jeff Tenenbaum, Managing Partner at Tenenbaum Law Group PLLC, on avoiding antitrust liability.
  • Case studies with the American Association of Colleges of Pharmacy, the Association of American Medical Colleges, the Casualty Actuarial Society, and IEEE.
  • A plain English review of key research terms, and a brief explanation of the rules of formal logic (and how they affect research work).
  • Recommendations for books, articles, websites, podcasts, and courses you can use to improve your research skills.
  • A series of thought questions for you to use to spark discussion with your team.
  • An extensive list of resources in case you want to dig deeper on any of the topics addressed.

Gardian of the Month: Kris Mayo

Kris Mayo is our Gardian of the Month! Find out her favorite thing about learning something new, networking advice for wallflowers, her go to resource and what she would title a book about her life!

Read More >
By: Kara Nacarato | Jan, 13 2023
Health

Workout For Good

​How are you going to make this your year to beat the odds and finally meet, and sustain, your workout goals? We’ve come up with a few simple tips to help you along the way.

Read More >

5 Facilitated Conversations You Can’t Afford to Put Off in 2023

As Aaron returned to a regular work schedule this week, it occurred to him that now’s the time to both reflect on our current level of facilitation knowledge and confidence, as well as take stock in those important conversations we need to have before we get too far into the minutia of the day-to-day meetings and tasks that are almost inevitably creeping into our calendars. Here are the five must-have conversations.

Read More >

Shaking Up Your Volunteer Strategy

The handwriting has been on the wall – associations have consistently failed to fully resource their volunteer systems.  We have in the past 2 years innovated our education and digital strategies, our decision-making models and our workplaces. To a certain degree, we innovated our member engagement strategy as well – and left the volunteer strategy to age another year.  Here's a new volunteer management model for 2023 and what it looks like.

Read More >

2023: Plan to Disrupt

2023 is the year leaders will benefit from disruption in the association sector. This post looks at three disruptions and how associations can turn discomfort into benefits to better focus on members, manage operations and invest wisely.

Read More >

Gardian of the Month: Yesenia Murillo

Yesenia Murillo is our Gardian of the Month! Find out what subject she'd like to learn more about, one of her favorite professional networks, the resource she thinks you should check out and which season best represents her!

Read More >