Blog FPO

Getting Smart About Research

This work is licensed under a Creative Commons Attribution-NonCommercial 2.5 License.

This guest blog post is by Polly Karpowicz, MBA, CAE, and Elizabeth Weaver Engel, M.A., CAE.

Association executives use research all the time. We run surveys and focus groups to better understand our members’ most important goals and most pressing problems so we can make good decisions about the programs, products, and services we offer to meet those needs. We sponsor industry studies to help our members and other audiences make good decisions about investing their resources wisely. We consult third-party (university, government, NGO) data sets and studies to understand the STEEP (social, technological, environmental, economic, and political) trends that influence the operating environments of the professions and industries we serve. 

The problem is that most of us lack formal training in research methods.

How can you be sure that the research you’re using or sponsoring is giving you the insight you need to make good decisions? How can you protect your association’s reputation as a trusted source of unbiased information for the profession or industry you serve?

You need to build your information literacy muscles.

That involves asking questions like:

  • How was the data the study uses collected? 
  • Who was included in the research? Who was excluded?
  • Who collected the data? Are those sources reputable? 
  • Does the study draw conclusions without data to support them? 

It also involves familiarizing yourself with the terminology of research (sampling methods, margin of error, confidence interview, statistical significance, data hygiene, different types of research design bias) and with your options for collecting and interpreting data and the pros and cons of each, for instance:

Quantitative research, which is designed to ask questions and collect responses numerically, with the goal of coming to the kinds of answers that can be described by levels of statistical significance and degrees of confidence. Surveys are the most common form of quantitative research, and they provide reassuringly specific answers. However, surveys are particularly susceptible to design flaws in either the questions, the answer options, or both that can bias the results.

Qualitative research, which is designed to gather data through conversation. Interviews and focus groups are the most common types of qualitative research. While qualitative results are necessarily impressionistic and anecdotal, they allow you to delve into subjects’ motivations, the why behind what they think, prefer, and do. However, interviewers walk a fine line between creating a high level of trust with interview subjects, which is necessary for success, and inappropriately influencing their responses which can bias results.

Does this all sound complicated? 

It’s important for associations to get this right, both so that association executives have the best possible chance of making good decisions about how to invest limited association resources to generate the best return for members, and because associations are viewed as trusted, unbiased sources of information for the members and other audiences we serve. It’s incumbent on us to provide quality research products so we remain worthy of that trust. 

You can learn more about these and other critical research concepts and read stories of associations doing great work in their research programs to serve their members and the professions and industries they serve in our new whitepaper, Caveat Emptor: Becoming a Responsible Consumer of Research, in which we tackle the sometimes thorny issue of what you need to know and do to be a savvy consumer and sponsor of research even if you DON’T have a formal background.

The monograph also includes:

  • An interview with Dr. Sharon E. Moss, co-editor (with Sarah C. Slater) of The Informed Association: A Practical Guide to Using Research for Results, on ethical practices in research.
  • An interview with Dr. Joyce E. A. Russell, The Helen and William O’Toole Dean at Villanova School of Business, on developing discernment in assessing research.
  • An interview with Jeff Tenenbaum, Managing Partner at Tenenbaum Law Group PLLC, on avoiding antitrust liability.
  • Case studies with the American Association of Colleges of Pharmacy, the Association of American Medical Colleges, the Casualty Actuarial Society, and IEEE.
  • A plain English review of key research terms, and a brief explanation of the rules of formal logic (and how they affect research work).
  • Recommendations for books, articles, websites, podcasts, and courses you can use to improve your research skills.
  • A series of thought questions for you to use to spark discussion with your team.
  • An extensive list of resources in case you want to dig deeper on any of the topics addressed.
By: Kara Nacarato | Nov, 18 2022
Best Practices

Considerations for Year-End Well Wishes and Gift Giving for Your Members, Customers, and Staff

As the holiday season and year end approaches, many organizations look for ways to acknowledge the season with their members, customers and staff. But this isn’t always an easy task and depending on your audience(s), coming up with the right messaging and/or gift-giving idea is crucial to ensure your well-intended gesture reaches your audience as imagined. Whether you are looking to say happy holidays, happy new year, or a simple “thank you” for your support, it is important that you take time to formalize the appropriate messaging and gift-giving options for your audience.

Read More >

Getting Smart About Research

Association execs are often tasked with choosing or even creating research on which our organizations will base important decisions, but many of us lack formal research training, which can be a little daunting! Fortunately, there's a new, free resource out that can help you develop your skills and confidence in research terminology and methods.

Read More >

In Event Planning…We’re All on the Same Team

In basketball, you win as a team, and you lose as a team. It doesn’t matter if you personally play your best game, if you aren’t supporting each other and working together as a team, you won’t be successful. This is true of events too! 

Read More >

Forces Impacting Our Success: MSAE Members Complete PESTLE Analysis

During the Michigan Society of Association Executives, members and attendees had the opportunity to participate in a facilitated exercise to identify and explore current forces at play that are affecting them and their organizations. Following is an overview of those sessions led by Event Garde team members, along with the strategic insights and actionable business intelligence association leaders can apply to their work immediately.

Read More >
By: Allison Laux | Aug, 12 2022
Associations

Findings of 2022 Non-Profit Survey on Value Proposition

Halmyre has released their latest report: Growing Value: Findings of Halmyre's Non-Profit Value Proposition Survey. This is based on the 2022 survey of Non-Profit Executives about value proposition, member engagement, pricing strategy, and data intelligence. Here are two highlights from the learning.

Read More >

Developing a Site Visit Itinerary

With sky high flight prices, travel delays, and lost luggage, it’s more important than ever to ensure every site visit is productive, efficient, and flexible. Here are 5 steps to guarantee that you are crafting a worthwhile itinerary for your group.

Read More >