Indexes and indicators are not synonymous: social and environmental indexes group specific sets of indicators to consolidate results and summarize the information contained in the indicators. As a result, indices allow global comparability across a wide set of categories that would otherwise be impossible to measure. In other words, when comparing individual indicators, we can observe that a given territory performs better or worse in relation to others. Indexes make it possible to group indicators to qualify territorial performance in global terms, quantifying the differences.
For this reason, socio-environmental indexes have been increasingly used by companies, government, private institutes and foundations to consolidate groups of indicators into a common metric allowing comparisons of territorial development processes over time.
A key condition of social progress is informed decision-making, based on reliable and up-to-date information. Indexes are tools used to simplify, quantify, and track technical information and to communicate it to various stakeholders.
The best socio-environmental indexes can be applied in many contexts and regions to facilitate comparison between territories, a factor that helps in communicating results and impacts. To convey credibility, indexes must be robust in their construction, and present descriptive justification for its composition, including collection methods and other technical choices, such as the selection of specific indicators.
Currently, hundreds of socio-environmental indexes are being developed and used in a wide variety of contexts. The best-known indexes for measuring territorial development are the HDI (Human Development Index), the IPS (Social Progress Index) and the IPM (Multidimensional Poverty Index).
Socio-environmental indices
​
Case: Social Progress Index
Partners: Fundación Avina, Klabin SA, IPARDES
SOCIAL AND ENVIRONMENTAL INDICATORS
Application case: Using Social Progress Index (SPI) in the company’s catchment area
Klabin S/A, Brazil’s largest paper producer and exporter, developed the PUMA II project, to substantially expand its packaging paper production capacity. The project’s impact area included 12 municipalities in the Campos Gerais region in western area of the State of Paraná. Klabin S/A needed a robust and comprehensive methodology to measure and monitor social development in the project’s area of ​​influence.
Anattá worked with state bodies and private foundations to obtain the SPI at the scale of the catchment area. The Social Progress Index (SPI) was carried out in partnership with Fundación Avina and IPARDES (the official statistics institute of the State of Paraná), among other actors.
​
Comparative historical data sets were produced for other, larger adjacent regions including municipalities from the Londrina and Ponta Grossa Regions (for comparison) based on administrative (public) data. The construction of this IPS data set was enriched with data obtained from specific surveys, including more than 1,500 interviews carried out with the regions’ residents.
Finally, the IPS was released on a dynamic platform, including annual update and monitoring functionalities. The Platform was released with the participation of mayors and secretaries of the main municipalities in the region, as well as with other companies and organizations in the territory.
Studies were conducted in specific regions of Western Bahia (BA) and Palmas (TO), with soy producers and rural communities to understand the impacts of soy agricultural production in the region known as Matopiba, which includes parts of the states of Piauí, Maranhão, Bahia and the state of Tocantins. The project was carried out within the scope of the Partnership for Good Development Project (GEF/PNUD), led by Conservation International Brazil.
In these studies, it was possible to monetarily quantify the impacts, both positive and negative, of soy production on the natural capital directly and indirectly involved in production (such as soil, water, etc.), as well as on the environmental services surrounding production (soil erosion control, climate regulation etc.). Furthermore, the main risks and opportunities arising from the relationship between production and the environment were identified.
​
Agricultural production practices of small rural communities, which are much more diversified in nature were also assessed. Potential risks and opportunities that may arise from these practices and the adoption of more sustainable management strategies were also identified.​
​
These studies were published by Imaflora and Conservation International Brazil.
Case: Application of the NCV Protocol to qualify soy production impacts on large rural properties and smaller rural communities.
Natural Capital Valuation
Environmental challenges are increasingly incorporated into the decision-making process of large investments, either by companies or by public authorities. Partially, this is due to the availability of incentives for the incorporation of sustainable practices in production processes. In this context, instruments such as Payment for Environmental Services (PSA), Green Bonds, incentives to produce clean energy, or simple alignment with the Sustainable Development Goals (SDGs), such as Zero Hunger and Sustainable Agriculture (SDG 2) fit into this context. Clean and Affordable Energy (SDG 7), Sustainable Cities and Communities (SDG 11), Sustainable Consumption and Production (SDG 12) and Action Against Global Climate Change (SDG 13) are other SDGs that have a direct relationship with corporate sustainability practices and should be taken into consideration when planning large investments.
​
External factors to production, such as CO2 emissions and effluent contamination, and the way in which these factors affect society and even production itself are difficult to measure and the methodologies are not standardized. The Natural Capital Valuation (NCV) Protocol comes in handy to overcome this challenge. The Protocol is an effective tool to quantify and qualify the effects of these external factors to production, thus improving the decision-making process.​
As defined by Capitals Coalition the “Natural Capital Protocol is a decision-making framework that enables organizations to identify, measure and value their direct and indirect impacts and their dependencies on natural capital”.
​
The Natural Capital Valuation (NCV) allows a company to understand and measure the impacts of certain forms of production and even estimate their social costs. Another benefit of NCV is to facilitate the understanding of how oscillations in the availability of a given natural capital can impact production, as well as to compare the different environmental results of different production practices.
​
The NCV is based on a set of steps that range from the identification of natural capital and environmental services related to production, to the definition of actions to be taken based on the information generated by the assessments.
Case: Soy production and agricultural production of small rural communities in the State of Bahia and Tocantins.
​
Partners: Imaflora, Conservation International
How can one ensure that the social development program managed by his or her organization produces consistent impacts over time, and in line with funders' expectations? How to mitigate the risks of the private social investment strategy that is being coordinated? How to show investors the results obtained by the company's social responsibility deparment?
The answer to all these questions is the same: through a good Results Monitoring and Evaluation System, which Anattá has called “SMART”, in allusion to the five SMART principles of indicator management (find out more here).
​
Monitoring and evaluation activities are fundamental for planning and executing actions to achieve the expected results. A results management system greatly improves the monitoring of indicators, generation of updates and reports, and communication with various stakeholders.
We understand that a good evaluation system starts with the human element. It is the team’s responsibility to observe and read the context as well to define te evaluation objectives. Once these objectives are well defined the system is then complemented and optimized with technology, which helps a lot in the production and dissemination of knowledge. SMART implementation must be preceded according to three steps:
MONITORING AND RESULTS EVALUATION SYSTEM
Case: Prosperity Fund
Partners: DFID
Context Analysis of the program: what are the specific characteristics, risks and conditions of success?
​
Theory of Change or Logical Framework: widely discussed and validated by the stakeholders.
Data Protocol: what information needs to be collected and with what frequency? Who collects and processes the data? How can technology support the system? How to be cost-effective?
1.
2.
3.
Discover the case of a simple and efficient management system built in a short time for a global program supported by DFID, the United Kingdom's Department for International Development, one of the largest funders of social development globally.
In this case, the management system (SMART) was built for a DFID program aimed at improving public health present in six countries, implemented in partnership with the federal government in Brazil. As the general lines of action of the program had already been defined internationally, it was possible to start activities by revising the Theory of Change for the program in Brazil.
​
Theory of Change validation workshops were held with the participation of program partners, divided into groups by thematic axis. Next, activity and product indicators, intermediate, result and impact indicators were produced, aligning the program's local planning with the project's global Theory of Change.
Finally, the periodicity and the most suitable sources of data to nourish the indicators were considered. We developed a lightweight system, based on the Microsoft Sharepoint architecture (a system already used by the client) with the possibility of providing access to external users who feed the database with questionnaires collected by cell phone. We developed database formats and specific forms for updating qualitative and quantitative indicators, which showed the program's contributions to improve health related public policies Finally, all results were made available through a dynamic website, with automatic updating of indicators, which was shared with the project’s funders.
Impact evaluation of a social program depends on choosing the best technique to assess with a high degree of confidence the verifiable changes attained by the program, and not by external factors. Much of the impact evaluation work is focused on understanding what changed and what was the program's level of contribution to the identified changes.
There are dozens of methodologies available, which must be chosen according to the evaluative questions, the available resources, and the time horizon. Impact evaluation should be carried out pragmatically and in accordance with the operational context of the programme.
Although the term “impact” is widely used in the universe of social program evaluation, the term must be used with caution: the impact is the social change that was generated by the programmatic actions and must consider methods to disregard changes (both positive and negative) due to external contexts, and which are therefore beyond the control of the program.
​
As an example, we can cite an income generation program in rural communities where, after two years of the program, the funding agency carried out an evaluation and actually detected an increase in income compared to the baseline. However, when comparing with other communities where the program does not operate, it was verified that income increased even more, a factor due to state income programs and new investments in the territory. That is, in this example, the impact of the program must be rethought considering comparison with other regions.
More complex concepts such as “control group”, “data handling”, and “counterfactual” are always used to better understand what the specific contributions of the program were, directly attributable to it.
​
We believe that impact evaluations should mix qualitative and quantitative elements, using narrative and econometric techniques, triangulating results to obtain clearer pictures of the results of the evaluated program. Impact evaluation should be carried out pragmatically and in accordance with the operational context of the programme.
Case: Evaluation of the Productive Bahia Program
Partners: world Bank
IMPACT EVALUATION
In this project, developed by the World Bank, the focus was on assessing the impacts of programs for productive inclusion as well the impacts derived from federal and state income transfer to rural areas in the State of Bahia. This project focused on four fundamental program development strategies, which are targeting, provision, coordination and integration and, finally, monitoring and evaluation of results.
​
Anattá used a mixed quanti/quali research design to collect and analyze data on ATER availability, distribution of Cisterns, access to Pronaf, Crop Guarantee, acquisition in PAA and PNAE, and other programs such as Produzir. Studies were also carried out of the socioeconomic conditions of beneficiaries of programs enrolled in CadÚnico. The results of the quanti-quali analysis, allowed for the composition of an in-depth analytical framework of the evaluated program.
The Sustainable Development Goals (SDGs) were ratified by the United Nations to guide development strategies for member countries. The SDGs consist of 17 objectives and 169 targets covering a wide array of dimensions related to sustainability. The goals were designed to be met by 2030, and as such are conventionally referred to as the “Agenda 2030”.
The SDGs are used by national governments, states, municipalities and, increasingly, by private stakeholders to guide investment allocations, prioritization of corporate responsibility responses and the width and depth of community development programs. The success of this approach depends on locating and prioritizing SDG goals and targets specifically at the territorial scale of development. To guide this effort, the Institute for Applied Economic Research (Instituto de Pesquisa Econômica Aplicada – IPEA) which has developed a detailed SDG assessment for Brazil, which can be found here https://www.ipea.gov.br/ods/. We at Anattá believe that the definition of specific SDG goals and adequate metrics depends on a deep and comprehensive territorial reading. This is fundamental for monitoring and evaluating State and corporate responsibility programs. Within this framework Anattá offers the possibility to develop indexes for performance evaluation, as well as the design of action plans, programs and projects aimed at meeting these goals. Our main product is the development of an SDG Acceleration Plan, which can be implemented through public policy actions, or multi-year plans (in the case of the state agencies), or social and environmental responsibility programs (in the case of private stakeholders).
Case study: Developement of the Piauí 2030
SUSTAINABLE DEVELOPMENT GOALS
The State of Piauí, Brazil, through the State Planning Secretariat (Secretaria de Planejamento – Seplan) and the United Nations Development Program (UNDP) signed a commitment to promote the acceleration of the SDGs in the State. Two consultants of Anattá’s had a role on this project, that was to lead the project coordination effort and to carry out the assessment of the Infrastructure Area.
The Piaui 2030 Plan was developed based on four dimensions: Environment; Labour, Employment, and Income; Infrastructure; and Reduction of Inequalities. Anatta’s approach defined three important stages in the delivery of the Plan. The first stage consisted of locating the SDGs according to specific scales of territorial development. This important step set forth the metrics to understand and accurately measure relevant goals and objectives.
During the second stage an in-depth longitudinal study was carried out for each dimension, to obtain a historical reading of the indicators, identifying the gaps (greatest distances) according to the established SDGs benchmarks.
After analyzing the indicators, the third stage consisted of an intensive field research component. Seplan's team of consultants and technicians visited the state's Development Territories, to verify the readings obtained from the previous phases as well as to identify the structural barriers to achieve the SDG benchmarks. Opportunities and potential strategies to accelerate the achievement of the SDGs were also identified.
The project generated a set of programs and action plans discussed with the local community.
These plans can be used by the State government to define priority areas, develop public policies and innovative programs aligned with territorial development needs. This alignment favors greater engagement of civil society, public authorities and private partnerships making the process more inclusive, economically robust and environmentally sustainable.
​
​
​
In addition to the Plan, a SDG index was developed by aggregating the respective indicators and the goals and indicators included in the State’s Pluriannual Plan. The State’s division of its areas into Development Territories, allowed the aggregation of indicators by group of municipalities that make up each of the regions. As such the implemented actions can be monitored according to an analysis of the aggregated indicators (by territory) guided by the particularities observed in each municipality. This tool supports the State’s ability to define priority areas and the needed investments to promote the acceleration of the SDGs.
​
This index was named the Territorial Development Index.
Traditionally, economic-financial modeling is used to evaluate the potential of the enterprise to reproduce the capital that will be invested in it. In other words, it is an instrument of financial capital, aiming at protect you from possible losses and preserve your profits. These models are usually developed and applied by technical teams that provide their services to the “client”. Unlike this approach, the Participatory Economic Feasibility Study (EPVE) applied by Anattá seeks to use the accounts of the economic feasibility study in order to allow entrepreneurs in the popular economy to take ownership of the tool and come to have a more coherent and profound knowledge of their own productive activities. At the same time, the Anattá approach seeks to eflect the collective and solidary organizational logic of the popular economy in numbers. And more: guided by trust and mutual respect between workers and their advisors, they recognize and want each other as companions who seek to build together a new path of financial autonomy and better quality of life. Thus, “in the search for numbers”, Anattá with the entrepreneurs ends up opening a space for “discussion of the relationships that govern the social organization of work” from the perspective of fair and solidary enterprise.
The accessibility and elasticity of the EPVE method, perfected by the Anattá team based on more than 15 years of experience with enterprises in the popular economy, is based on a financial instrumentation process, based on seven basic arithmetic equations. The simplification of accounts seeks, ultimately, to ensure the real appropriation of the tool by the enterprise and its complete autonomy without necessarily depending on other specialists.
With this objective in mind, the EPVE methodology seeks to answer the three fundamental questions:
Case study:
Participatory Economic Feasibility Study
Anattá's technical team has more than 15 years of experimentation with EPV, applied in several segments of the popular economy, from agriculture and family fish farming, to beekeeping, the production of various handicrafts, as well as the food and tourist segment. The diversity of the tool's application attests to its elasticity and modification according to the specific demands of the productive communities.
We work in partnership with several NGOs and Foundations, meeting the demands of productive communities including family farmers, artisanal fishermen, caiçaras and quilombos. At Quilombo Fazenda da Caixa, on the north coast of São Paulo we built the Community Restaurant EPVE integrated with environmental guide services, ready to receive tour groups. We also applied EPVE to improve the flow and sale of juçara pulp by caiçara producers in the Ubatumirim Community in Ubatuba.
Outside Brazil, in Bolivia. Anattá worked with a consortium of non-governmental organizations dedicated to strengthening the fish farming sector. Using resources from the Canadian Fund for Food Security (IDRC), Anattá worked with family fish farmers, training a network of extension workers in EPVE techniques. This intervention helped the fish farmers to identify the impacts of new technologies, applied in the management of the tanks, on the economic viability of the undertakings, increasing the production of fish and contributing to greater food security in the territory.
1.
2.
3.
Ultimately, we understand that these three questions can only be answered when we see that the sustainability of Popular Economy Enterprises is not a technical (only) or strictly economic issue, but essentially a political one. Economic viability therefore depends on political actions committed to a process of social transformation. To this end, the Anattá approach centralizes the social learning process, always attentive to the search for possibilities for change. It is a movement, above all, a field under construction and experimentation.
relationship between work process, economic viability and way of managing associative undertakings;
​
relationship between the individual and collective scale as well as the alliances needed to build and/or strengthen marketing channels
​
relationship between the sustainability of the Popular Economy and the political project of territorial development.