How do we define impact?
Our central calculation
Total Impact = (Economic impact + Social impact + Environmental impact)
Expand for defintions
There are multiple aspects of impact to consider. At Raw Material, we are concerned about our impact across three main dimensions, weighted in this order:
1. Economic impact
Enabling economic freedom for people working in coffee. In particular, those for whom coffee is a large proportion of their household income. We focus where improvement in income levels, volatility, or payment timing will most likely have a significant influence on wellbeing. Enabling this economic impact through an improved system is the core focus of our work, measured at both the household and community level.
2. Social impact
Contributing to the development of and sharing of knowledge across all participants in the coffee sector, to enable improved inclusive income opportunities and community wellbeing. For example, working with producers to improve production practices; training cuppers and graders in remote regions to recognise quality early and empower marginalised groups; working with roasters to adopt buying practices that include stable prices and long-term planning.
3. Environmental impact
The long-term sustainability of the coffee sector, both economically and environmentally. Through our work, we help develop and share environmentally sustainable practices. These are designed in the context of local constraints, including transition, hidden, and ongoing costs, in order to improve viability and the likelihood of long-term adoption.
IMPACT
Theory Based Impact Evaluation
It is vital that we understand our effectiveness; to find what works and do more of it.
However, to be able to expand and reach more communities, we need to understand the reasons for the outcomes observed and the circumstances under which similar results are likely to be replicated. Theory-based impact evaluation involves mapping out the causal chain from inputs to outcomes and testing the underlying assumptions.
Through Raw Material, roasters buy from smallholder producers in Colombia, Burundi, Rwanda, Mexico, and Timor-Leste
Each country has a distinct situation to consider, as does each region, association, and individual. In order to be effective in this range of contexts, what we do and how is tailored; designed to fit each environment.
Mauricio Benavides the Legal Representative of El Carmen de Acevedo, receiving roasted coffee from roasteries who support the Red Associations during the annual meeting.
RM Timor-Leste, Ameta's team turning the coffee of our first export during the second drying phase in Railaco, Timor-Leste
We follow the key principles of the theory-based approach as outlined by The International Initiative for Impact Evaluation:
1. Map systems & test assumptions: Map out the causal loops to explain how the interventions are expected to lead to intended outcomes. Test the assumptions that underpin the causal links and update approaches to account for new understanding.
2. Understand context: Strategies should always be considered in the correct context, including the social, political, and economic settings.
3. Evaluate impact using a credible counterfactual
4. Use a combination of quantitative and qualitative methods.
IMPACT
Counterfactual Analysis
What is a counterfactual?
To synthesize a meaningful statement about impact, it is important to compare the outcomes measured not only with the past but also with the most likely alternative outcome. This is the counterfactual: “What would have happened if we weren’t active in a region?”. The difference between the most likely counterfactual and what happened is a logical measure of directly attributable outcomes.
A significant challenge in evaluation is that the counterfactual cannot be directly observed, it must be approximated. Commonly this is done with a comparison group. Increasingly we use other techniques that are more efficient and better address issues of selection bias, spill over, and contamination.
Don Chepe and his family, during the first meeting between Raw Material and El Carmen. Huila, Colombia
Producers, community members, and roasters meeting for the annual gathering at El Carmen
Example in action:
El Carmen, Huila, Colombia
We don’t compare the price of coffee sold via Raw Material in El Carmen with a reference community. Instead, we compare El Carmen’s sell price with the price offered across the same period by the buyer the community sold to in the past. As long as this buyer remains the alternative option, this price is the counterfactual. To gather this changing data efficiently, we don’t record it each day. Instead we capture the price several times across the season and compare it with the daily Precio Interno de Referencia.
This allows us to deduce the equation the alternative buyer is applying to the national reference price, to find the parchment price they offer. With the resulting high-fidelity curve of counterfactual prices across time, we can calculate improvement accurate to individual producer and time of sale.
The annual meeting of producers and roasters of Red Associations, El Carmen, Huila.
IMPACT
Data Collection
How do we collate the data to measure our impact?
Quantifying, collecting, and analysing impact metrics is a valuable activity, but it comes at a significant cost. Every dollar or minute spent measuring is time and money not invested in something potentially more valuable. It is important therefore, that we know what we want to measure and why, and build a plan to minimise the cost of data collection, and free up maximum resources for other investments.
Qualitative data can be particularly useful in understanding the reasons something is working, and therefore, how an intervention could be replicated in other settings.
Pedro Lopez Perez and Carlos Perez-Santos, producers from Tierra Blanca with Salomon in San Pedro El Alto, Sierra Sur, Mexico
Coffee producers in Ermera, Timor-Leste, attending processing training courses with Miguel of Raw Material Colombia
To gather qualitative data we use focus groups, interviews, and field visits.
We must collect information that will allow us to infer change across the range of the most important factors (economic, social, environmental) through careful use of proxies, and deliberately choose to measure no more than that. Local circumstances will dictate which metrics are most useful and possible to measure.
The information we gather should be detailed and accurate enough to inform these choices:
1. What is working?
2. What isn’t working?
3. Where?
4. Why?
5. And for how much?
Raising quality and shelf-life, through low-cost interventions
In Colombia, our core focus is to raise quality and shelf-life through low-cost interventions, working with the existing structures. We do this by creating systems to ensure consistency of large lots at a community level. This in turn enables reliable, stable, and improved household incomes for smallholder farmers through connection with roasters. Our secondary focus is to shift the power dynamic in the coffee sector; as specialty becomes a norm, there are new opportunities for women to gain higher-income and higher-powered roles. The third area of focus is environmental; Colombia has a high average output per hectare, but this comes at the costs associated with mono-cropping and high-input farming techniques.
We don’t invest much at community level infrastructure in Colombia, compared with Timor-Leste, for example. In Colombia, groups of small farms working as a team to reach the specialty market with large, consistent lots year-round is the key to success, with the majority of funds reaching households rather than stopping at the community level.
To improve household incomes, outcomes for women, and to encourage a shift towards more environmentally sustainable farming practices, in Colombia we:
1 Set up cupping labs, and focus training towards women in the community
2 Build and join community associations, invest in infrastructure and training to improve quality, and take their coffee to market
3 Place QC staff and equipment at parchment buying points to improve consistency
4 Consult wet mill operators and producers in-person on best practices
5 Purchase for consistent prices based on what coffee costs to produce, and what roasters are willing to pay to maximise profitability for producers
Structural Payments
The minimum price paid to a producer working with Raw Material in Colombia is 1M COP/carga of parchment coffee. Across the country we have teamed with individual producers, as well as co-operatives, and associations. The largest volumes of Raw Material’s coffees come through the relationships that we have built in areas with communities looking for a solution to low, fluctuating prices.
When a sample is confirmed by a client, by any of our importer-customers, or by an RM distributor that will import, store, and finance bags for local roasters, RM Colombia then receives the parchment from the producer. The local market price is then paid when the coffee is unloaded at the dry mill, once the physical and sensorial qualities have been checked.
Olga Lucia Botero of Café y Procesos, our QC partner in Colombia
Community cupping at El Carmen, Huila, Colombia
The lots are then sorted, graded, selected, packed, cupped, and exported. When the coffee is paid at FOB, RM Colombia pay the difference between the first payment and the 1M COP/carga.
To ensure that the producer receives the correct payment for the coffee sold, (plus a potential additional third payment dependent on quality), we partner with a local dry mill, to build correct records of those who have sold coffee to the mill lot by lot, to ensure payments are traceable and correct.
Timor-Leste is a young nation, where coffee has become the most vital crop to secure the country’s future.
In 2002, Timor-Leste became the first new independent state of the 21st century. This was preceded by 25 years of conflict that had claimed 200,000 lives and destroyed all infrastructure, pushing most into extreme poverty. Currently, Timor-Leste is the country most reliant on oil for income, a rapidly depleting resource. Coffee is their most important and profitable crop, and is the nation’s second largest export.
Jose Dacosta and others, preparing bamboo for the construction of the Raimutin wet mill, Ermera, Timor-Leste.
Oil has funded much of the country’s development, but stocks are depleting. Fast.
The importance of coffee to thousands of smallholder families cannot be overstated. Approximately 77,000 families depend on coffee for income, impacting approximately 37.6% of all households. A 2011 survey of over 800 households (5,300 individuals) conducted in Ermera district reported that more than half of the households surveyed rely on coffee for the majority of their income. More than a quarter surveyed stated that they received more than 90% of their income from coffee.
Participants in the coffee quality training program, hosted by Miguel of Raw Material, Colombia
With the end of oil income in sight, improving the coffee sector is of critical importance right now.
Presently, however, the productivity of coffee as a crop is extremely low due to several factors (most notably, aged trees reducing yields, and questionable land tenure). The profitability of the crop for a producer is both low and volatile, as almost all coffee is sold in the commodity market at a discounted C price.
With that fact in mind, 37% of Timor-Leste’s households depend on coffee for income, leaving them in a position of vulnerability in a market they are soon to vitally depend on.
We aim to achieve goals in ways that are low-cost, easily replicated, and through initiatives that ensure the results are captured directly by the most marginalised.
Raw Material’s aim is to help improve the total value and profitability of Timor-Leste’s coffee sector, in the most efficient and direct ways possible.
Most of our activities can be grouped into one of three buckets; raising coffee quality, increasing productivity, and building demand in markets that offer higher, stable prices.
Ameta of RM Timor-Leste, discussing the altitude variance across Ermera District, Timor-Leste
The Potential of Socio-economic Impact
We use a framework to compare options of where and how we work in terms of potential for economic and social impact. Following our analysis of the market in late 2017 and early 2018, it became clear that Timor-Leste's situation measured very high in terms of scale, neglectedness, solvability, and personal fit.
In response, we created an accelerated plan for the high-stake situation. We met with groups of producers in villages across Atsabe to hear how the coffee market has served them in recent history. Together we developed a plan for a prosperous future through:
- Sustainably raising and maintaining the quality of Atsabe’s coffee predictably above 80 points
- Connecting with roasters who want to commit long-term
- Systematically solving the diverse challenges on the path to selling in the specialty market; financial, legal, cultural, infrastructural, technical etc.
Community meeting at the Atsabe wet mill, Timor-Leste
Burundi, like Rwanda, has centralised buying points for producers in the form of coffee washing stations.
At these buying points, coffee is sold in its cherry form to the station, where it is processed, dried, and warehoused for sale. In this way, we work directly with the washing station themselves, and provide premiums through the purchase price, which will increase the producer’s profits.
Washed coffees drying at the Izuba wet mill, Burundi
Nyagishiru Washing Station, Burundi
Our wet mill, Izuba, is a community washing station in the Kayanza region.
We purchase cherries from smallholder producers in the area, and then process the coffees as washed, honey, and natural lots. Through Izuba, we guarantee the prices paid to producers, and increase the value of their coffee through high attention to detail and quality control. This ensures the highest quality coffee and greatest profitability for producers.
Last year, almost none of the households around Izuba washing station had health insurance. In Burundi this season, we have focused on ensuring all families delivering cherries to Izuba receive full health insurance. Following interviews with the community, it was revealed that this initiative was the most wanted and needed addition we could provide.
Our partner in Rwanda is Muraho Trading Co.
Muraho Trading Co operate coffee washing stations across several areas of coffee production in Rwanda, working with the specialty coffee market in mind, and the potential that it has to expand the profit producers can achieve. This is done not only through premiums on top of the local market rate, but also through the expansion into honey, and naturally processed coffees, to expand the volume that can be purchased and processed at a station. Their considerations on where to work have also increased the impact that they can achieve, by working in areas that might be low-volume coffee regions, where producers are in need of neighbouring stations for a point of sale.
Ayingeneye Marceline, Usaniwabo Christine, Usabimana Evariste, Kamariza Jacqueline, Nakure Josiane, and Nyirasugira Beatrice working at Shyira coffee washing station.
Gisheke Washing Station, Rwanda
Example in action: Shyira Coffee Washing Station, Nyabihu District
Shyira’s washing station was built to expand the selling potential of the co-operative of producers that live and work around the town of the same name.
Shyira created many jobs for the surrounding community and provides many options for income generating activities. Young adults in the community are regularly employed; from the construction of the wet mill, through to the coffee harvests every season.
Women make up 75% of the station’s staff. Their employment brings a second income stream to their household. This improves general living conditions and quality of life for families who work with the washing station.
Farmers have been provided with the knowledge, equipment, and training of better agricultural practices in order for them to get better yields from their trees. Further to this, a total of 55,000 seedlings were nurtured and distributed to local producers. On top of this, they provided a “Mutuelle de Santé” (local health insurance) for all the farmers who supply the station.
Ensuring the long term viability of coffee production in Southern Mexico.
The challenges faced by coffee producers in Mexico are not unique. In Oaxaca, a decade of coffee leaf rust disease coupled with a systemic lack of support and infrastructure has reduced production yields by up to 90%. The average producer now has just 100kg of parchment to sell each year, making coffee production fundamentally unviable.
The inevitable effect of this is a surge in rural migration to urban centres, a sharp increase in the average age of coffee producers and an overall reduction in the amount of people growing coffee. These are problems common to many coffee producing regions across the globe, but remain as acute in Oaxaca as anywhere in the world.
Doña Lucia, harvesting coffee at Finca Muxbal, Mexico
Developing QC and engaging younger generations.
Community level investments will take time in Mexico. Our work here is in its infancy, associations are loosely organised, and building trust and relationships takes time in regions that have been ignored for decades.
An early area of focus is developing basic QC facilities in both the Sierra Juarez and Sierra Sur. Providing physical QC equipment such as mini-hullers and moisture meters allows for a base level of QC at farm level, one of the quickest routes to increasing coffee quality. In time, we aim to expand these facilities to incorporate cupping labs and sensorial analysis. Following the models in Colombia and Timor Leste, cupping labs and QC programs should be developed and run by women and younger members of producing families.
Increasing yields and developing organic markets
The long term goal in Oaxaca is to increase production yields through managing coffee leaf rust, providing access to agricultural inputs and making coffee profitable enough to encourage stumping and replanting programs.
The traditional farming style in Oaxaca is a low impact, organic system of production that leaves very little footprint on the local environment. This means there are many organic certified producers in Oaxaca.
Developing the market for organic specialty coffees from Oaxaca will allow us to work with more producing families and to quicker generate funding for increasing production yields and profitability for all.
Romulo and Alejandro of Paraje Los Machos, with Al and Miguel from Raw Material and Shaun from Red Beetle Coffee Lab
Our aim is to stem the tide and make coffee a long term, economically viable proposition. This is for current and future generations, through higher prices, and a quicker return on investment.
Our initial strategy is to match the total counterfactual price local producers expect to receive from the local market in the first payment (made upon delivery of parchment). A second payment is then made upon export of the coffee based on the quality and final sale price. Providing higher and quicker returns on investment ensures our system is always the best option for producers, regardless of cash flow concerns.
81 - the number of households reached in 2020
150 - projected households reached in 2021
70% - Percentage of the total purchase price of the coffee made as a first payment
2.92 – the average increase in profitability for producers selling their coffee through RM Mexico supply chains.
Access to finance for these first payments is a limiting factor for us in Mexico currently and we are working on increasing our capabilities as this is the most immediate impact we can have in Mexico.
