We recognize that the roles of evaluators are complex, and must continue to adapt to a changing world.  As a foundation representative expressed during a convening several years ago, “I feel for evaluators. They need to be experts in all things practical and conceptual. Facilitation, data collection, data analysis and interpretation, equity, relationship management, and therapists too!” So, what are the roles of an evaluator? What do our clients expect? And how can we continue to use our position to advance equity and social change by practicing equitable evaluation?

In our session, we asked participants to reflect on the five roles and consider the question: What can the evaluation professional do better or differently—given our current context and where we are today in the field? These roles reflect the various hats an evaluator must wear and the responsibilities they carry for the community, the client, and the field. Following the EERS session, we posted our questions on Twitter and continued the conversation.

In the section below, we share our perspective on the five roles, and incorporate insights from the session participants. We hope this is just the start of an ongoing dialogue on how we collectively envision and effectively carry out our roles as evaluators. And we would like to know: Did we get it right? Did we miss anything? How can we describe the roles of evaluators even better?

Evaluators often begin a project by serving as an Expectation Setter—managing the often-broad hopes for systems change that both clients and evaluators bring to an engagement. While funders might desire wholesale systems change, we need to remember that change is slow and rarely linear. How do evaluators help make funders’ expectations more attainable? We define goals and expectations, set boundaries, and keep the evaluation reality based. Setting parameters for the evaluation up front is critical to ensure the project is doable in the appropriate time frame, meets the client’s learning needs, and produces findings that are actionable—all while maintaining a focus on values. In addition, the start of a project is the time to begin creating rapport with the client and build the foundation of a long-lasting professional relationship. And with all these plans, the evaluator must also remain flexible— staying grounded in a plan, but ready for changes based on emerging ideas and information.

“Clearly setting expectations helps me meet the needs of clients, which in turn helps clients achieve their objectives. When clients achieve their objectives, I feel like I have achieved mine.” – EERS Session Participant

Evaluators must also be aware of power dynamics among partners (e.g., funders, grantees, and communities), and help navigate these dynamics by naming them with the intent to shift power through collaboration and communication. We are operating in an historical context in which evaluation has sometimes caused harm to the communities it examines, and we must strive to overcome this past harm by setting and defining new expectations around equity, anti-racism, and supporting community voice and decision-making.

Once the project is underway and data collection begins, our role as Translator emerges. We gather data, observe dynamics, and make sense of them—bringing our findings back to the community to test them and see if we “got it right.” We have an opportunity to address equity through reflecting on data interpretation with evaluation participants, ensuring that the messages we pull from the data resonate with the community of origin and appropriately represent their voice and their priorities. This approach can lead the funder toward a more equitable program strategy based on these community strengths and needs. Above all, we need to use data for learning and change. For our findings to matter, we must present them in a way that represents community priorities and pushes our client toward improved action. Acting as a translator means understanding the story behind the data and communicating meaning for the funder, grantee, and community, to influence the systems we aim to change.

“I think this is really what it’s all about: translating data into actionable recommendations. Evaluation is useful when it’s understandable and actionable.”  – EERS Session Participant

As Learning Partner, evaluators help our clients and their grantees respond to the data and adjust their strategies. We learn and share during each stage of the evaluation, but most critically when using data to support action and make decisions. While evaluators may be positioned as external experts and partners, we must be comfortable as a co-learner, understanding trends and processes together with our clients across the course of the evaluation. Learning goes both ways, and is one of the best methods to dissipate a power imbalance and build trusting relationships.

Learning can sometimes conflict with the logistical aspects of our work, but as a learning partner, part of our role is to help funders reflect on how their practices might contribute to inequitable policies. We must understand that sometimes learning can be uncomfortable. Tight budgets may force us to aim for efficiency, sacrificing the time to explore data more deeply with a learner mindset. Our goal must be to always preserve room to reflect, propose new ideas, and to change and adapt—remaining curious and retaining our learning approach.

“[This is] the happy surprise of learning alongside the client/stakeholders: I love to uncover and co-interpret what data is saying, especially when it’s unexpected or elevates something long buried.” – EERS Session Participant

As Knowledge Sharer, evaluators abide by the notion that “what is learned here, leaves here,” and we must find ways to package those learnings in ways that can be taken up by our clients, the community, and the field. In this function, we must again recognize our positional power as we interpret data and ensure that the community remains the owners of their own stories. We are driven by the value that mutual knowledge sharing—along with learning and authentic, respectful translation of stories that are resonant and useful—may lead to thriving communities.

Finally, as Fiscal Steward, we must manage the client’s funds with integrity. We often are faced with conducting complex, ever changing evaluations with a fixed budget that may not leave room for the deep work that we know would strengthen our understanding. These are business challenges; as one session participant noted, “The evaluation marketplace can be a race to the bottom in terms of pricing, making it tough to meet client needs and stay viable.” These are also equity challenges, as evaluators and funders must be held accountable for these resources by serving the community’s needs. We must be mindful of the power dynamics that surface around money, and avoid adopting a “charity mentality” of “helping” or “saving” the community rather than working with the community. Community members are the experts and, as such, we need to compensate them for their time and shared knowledge.

As evaluation practitioners, it is in our nature to grapple with big questions, to lift up layers and peek underneath as we search for meaning and work with communities to identify solutions that might create a more just and equitable world. We are driven by a desire to improve our world through changing systems, identifying and addressing systemic inequities, highlighting community assets and priorities, and sharing learnings to facilitate that change. As one individual shared on Twitter following the session, evaluators can function best as a “change agent.” It is our hope that by better understanding what we do, in a way that is responsible, transparent, and grounded in the community, we can further move the needle toward systems change.

So, as we noted at the top, what do you think? Did we get it right? Did we miss anything? How can we describe the roles of evaluators even better? Email us at communications@equalmeasure.org, or join the conversation on Twitter @EqMeasure.

 

 

 

Continuing the Dialogue

• In a follow-up conversation about the blog, one attendee asked whether the goal of evaluation should be to permanently change power dynamics so that funders give communities full say over what to do with the money in their communities.
• There were several ideas generated from follow-up conversations on Twitter:
  • One user suggested “Values Clarifier” as a role similar to Expectation Setter.
  • One suggested “Assumption Challenger,” “Creator/Co-Creator of Knowledge,” and “Perspective Sharer/Broadener” as roles similar to Learning Partner.
  • One suggested “Facilitator” as a role similar to Knowledge Sharer.