Blog FPO

Getting Smart About Research

This work is licensed under a Creative Commons Attribution-NonCommercial 2.5 License.

This guest blog post is by Polly Karpowicz, MBA, CAE, and Elizabeth Weaver Engel, M.A., CAE.

Association executives use research all the time. We run surveys and focus groups to better understand our members’ most important goals and most pressing problems so we can make good decisions about the programs, products, and services we offer to meet those needs. We sponsor industry studies to help our members and other audiences make good decisions about investing their resources wisely. We consult third-party (university, government, NGO) data sets and studies to understand the STEEP (social, technological, environmental, economic, and political) trends that influence the operating environments of the professions and industries we serve. 

The problem is that most of us lack formal training in research methods.

How can you be sure that the research you’re using or sponsoring is giving you the insight you need to make good decisions? How can you protect your association’s reputation as a trusted source of unbiased information for the profession or industry you serve?

You need to build your information literacy muscles.

That involves asking questions like:

  • How was the data the study uses collected? 
  • Who was included in the research? Who was excluded?
  • Who collected the data? Are those sources reputable? 
  • Does the study draw conclusions without data to support them? 

It also involves familiarizing yourself with the terminology of research (sampling methods, margin of error, confidence interview, statistical significance, data hygiene, different types of research design bias) and with your options for collecting and interpreting data and the pros and cons of each, for instance:

Quantitative research, which is designed to ask questions and collect responses numerically, with the goal of coming to the kinds of answers that can be described by levels of statistical significance and degrees of confidence. Surveys are the most common form of quantitative research, and they provide reassuringly specific answers. However, surveys are particularly susceptible to design flaws in either the questions, the answer options, or both that can bias the results.

Qualitative research, which is designed to gather data through conversation. Interviews and focus groups are the most common types of qualitative research. While qualitative results are necessarily impressionistic and anecdotal, they allow you to delve into subjects’ motivations, the why behind what they think, prefer, and do. However, interviewers walk a fine line between creating a high level of trust with interview subjects, which is necessary for success, and inappropriately influencing their responses which can bias results.

Does this all sound complicated? 

It’s important for associations to get this right, both so that association executives have the best possible chance of making good decisions about how to invest limited association resources to generate the best return for members, and because associations are viewed as trusted, unbiased sources of information for the members and other audiences we serve. It’s incumbent on us to provide quality research products so we remain worthy of that trust. 

You can learn more about these and other critical research concepts and read stories of associations doing great work in their research programs to serve their members and the professions and industries they serve in our new whitepaper, Caveat Emptor: Becoming a Responsible Consumer of Research, in which we tackle the sometimes thorny issue of what you need to know and do to be a savvy consumer and sponsor of research even if you DON’T have a formal background.

The monograph also includes:

  • An interview with Dr. Sharon E. Moss, co-editor (with Sarah C. Slater) of The Informed Association: A Practical Guide to Using Research for Results, on ethical practices in research.
  • An interview with Dr. Joyce E. A. Russell, The Helen and William O’Toole Dean at Villanova School of Business, on developing discernment in assessing research.
  • An interview with Jeff Tenenbaum, Managing Partner at Tenenbaum Law Group PLLC, on avoiding antitrust liability.
  • Case studies with the American Association of Colleges of Pharmacy, the Association of American Medical Colleges, the Casualty Actuarial Society, and IEEE.
  • A plain English review of key research terms, and a brief explanation of the rules of formal logic (and how they affect research work).
  • Recommendations for books, articles, websites, podcasts, and courses you can use to improve your research skills.
  • A series of thought questions for you to use to spark discussion with your team.
  • An extensive list of resources in case you want to dig deeper on any of the topics addressed.

Benefits of Professional Accountability Groups

Accountability Groups are becoming common practice - they can serve lots of different functions and mean lots of different things to different people. They are any group of like professionals who want to provide support for, provide personal and professional growth opportunities for, and build relationships with others in their field. Read on for the benefits of these groups.

Read More >

My Favorite Things - Event Planning Edition

As we come to the end of 2022 events, I feel tired, exhausted, and ready for a break. But looking into 2023's schedule, I am reminded of a few of my favorite things…so come sing along!

Read More >
By: Kara Nacarato | Nov, 18 2022
Best Practices

Considerations for Year-End Well Wishes and Gift Giving for Your Members, Customers, and Staff

As the holiday season and year end approaches, many organizations look for ways to acknowledge the season with their members, customers and staff. But this isn’t always an easy task and depending on your audience(s), coming up with the right messaging and/or gift-giving idea is crucial to ensure your well-intended gesture reaches your audience as imagined. Whether you are looking to say happy holidays, happy new year, or a simple “thank you” for your support, it is important that you take time to formalize the appropriate messaging and gift-giving options for your audience.

Read More >

DEIA Data Collection: Overcoming the Hurdles to Advance Our Missions

One of the questions organizations struggle with regarding diversity, equity, inclusion, and accessibility (DEIA) is, “How do we measure our efforts to ensure we are making the impact we are striving for?” Data-driven decisions are a good way to eliminate bias in any situation. There are research studies out there demonstrating the fact that successful DEIA efforts do lead to more innovative, productive, and higher earning organizations over their competitors. To share those types of results and narratives, we must have the data to support them.

Read More >
By: Aaron Wolowiec | Nov, 4 2022
Meetings

Redesigning Association Meetings for Attendees Fundamentally Changed by the Pandemic

Due to the pandemic, it should be no surprise that our business-as-usual, pre-pandemic meetings are going to miss the mark. Following is an overview of just some of the ways our attendees are different, why it matters, and the tangible design edits you might consider for your upcoming meetings and events to better meet these new and evolving needs and expectations.

Read More >
By: Kara Nacarato | Oct, 28 2022
Meetings

7 Event-Planning Lessons Learned in 2022

Many of us are back to planning in-person events and while we continue to use our experience to help deliver first-class events, not everything is as it once was when it comes to event planning and execution. Here are some of the key lessons Kara will take with her into 2023.

Read More >