D E V E L O P M E N T

Evaluation: Who are you to judge me?

While just the thought of being evaluated can instill fear in even the most confident organizations, the process usually does more good than harm by teaching people how to more effectively attain their goals. It can also bolster confidence by showing an organization what it is doing right.

BY DIRK AMTSBERG

  Evaluation is a word that sounds horrible to most of us. The mere sound calls up images of hard-nosed officials or outsiders ready to criticize us. We fear bad reviews, not just because we do not deserve them but because an outside evaluator could not possibly know us well enough to really judge us. Often we even suspect the whole evaluation is simple an instrument to close the institution without giving us the opportunity to improve our performance.

  Despite the fear, this normally is not the case at all. For sure, evaluation is a device to gauge the success of an organization or organizational unit in respect to its vision, goals and tasks. But the primary goal of most evaluations is improve performance. The process is a very helpful way to shed light on the problems we have in reaching our goals. And knowing the problems is the starting point for improvement. An evaluation can also build confidence by underlining and organization's strengths and successes - another reason self-evaluation is useful and should take place on a regular basis.

  Finally, comparing evaluations from similar organizations can show multiple ways of solving similar problems. Different constraints arise in different surroundings, but in many cases other groups with goals similar to our own will think of creative solutions we can adapt to our own work.

A tale of two EMTCs

  The Environmental Management Training Centers in Central and Eastern Europe (EMTCs) began in November 1992 when the Bulgarian EMTC opened its doors. The Polish EMTC opened next, in March 1993. Both centers were established by the Institute for Sustainable Communities (ISC) and the U.S. Environmental Protection Agency (EPA) in collaboration with local institutions.

  The major purpose of an EMTC is to provide training and technical assistance in fundamentals of environmental management to managers and decisionmakers at all levels of national and local government, NGOs, universities, media and the private sector. The Bulgarian and Polish EMTCs are the oldest of the EMTCs established by the U.S. EPA, and that is why they were chosen for the first EMTC evaluation report.

Good evaluation results

  It is important to stress that both EMTCs got good marks. Both have an excellent reputation among local organizations and international donors. The Bulgarian EMTC was even named the leading environmental training institution in its country.

  The institutes disseminated Western knowledge in their countries and in this way formed a basis of experts who have gone on to improve the quality of environmental trainings (about 1,000 in Bulgaria and 2,000 in Poland). By educating about 100 facilitators in each country, the centers enhanced the self-training abilities of their countries. Training deliveries to state agencies, businesses and society increased the awareness for the need of environmental management and furthered democracy.

Differences between the two centers

  The division of labor between board, directors and trainers was exemplary at the Bulgarian EMTC. The board is responsible for the development and implementation of the EMTC strategies. All board members were actively involved in the organization's activities and promoted the center where they could. The relationships between director, board and staff are also excellent. Therefore, the director can concentrate on implementation and management of day-to-day activities.

  The Polish Board, on the other hand, was mainly supervising the activities of the EMTC while the director took over many of the board's responsibilities. This did not seriously hinder the activities of the EMTC, but it is still unfortunate that the board members do not support the organization more actively.

  Further differences between the two centers mainly concern the cooperation with international donors, the level of self-sustainability, and the level of adaptation of the training courses. All these differences stem from one major factor: the different environment in the two countries.

  The strive for sustainability of the Polish EMTC through the collection of fees for their courses was only possible because the Polish market is a lot larger than the Bulgarian. In Poland, businesses and NGOs have more money, and they recognize the need for environmental training. The Polish EMTC therefore was able to begin its "commercial courses." The EMTC began by introducing small fees that were gradually increased over the following years. The fees now completely cover the costs of the courses and will soon become the organization's main source of revenue.

  The Bulgarian EMTC did not have the possibility to follow this track because the economy is too weak and because the target groups do not yet see the necessity for environmental training. This situation forces the EMTC to rely a lot more on grants, and it depends upon international organizations than the Polish EMTC. It may still be possible, however, to collect a symbolic fee, a step that could help prepare training participants to pay their own way eventually.

  With funding for trainings arranged beforehand from outside sources, the Bulgarian EMTC has had more freedom to adapt their courses to the Bulgarian reality and to develop new courses they regarded as necessary. On the other hand, clients in Poland, who help determine which courses will be offered by only paying for subjects that interest them, wanted to have the certificate of an EPA course. Therefore, the center delivered only EPA courses with limited or no adaptation.

Key elements for success

  The cooperation of the EMTCs with the EPA was one of the key elements of their success. The EPA provided high-quality training and well-organized training materials.

  Another important element was the "customer orientation." Both institutions reacted differently to the challenges of their specific environments, for example in the level of adapting EPA materials to satisfy clients needs.

  The evaluation report further states the EMTC Network was important to the development of the centers by bringing new ideas and further prospects for sustainability. But the evaluation states that an improvement of the communications with the other EMTCs would be very helpful.

RECOMMENDATIONS FOR THE FUTURE

Development and future direction of a training organization depend upon its specific environment. However, here are a few general recommendations from a recent EMTC report on areas in which training organizations could improve in the future:
  • Marketing and promotion. So far, none of the EMTCs has developed a marketing and promotion strategy. This will become more important as donors will provide less money for Central and Eastern Europe in the years to come.

  • Less theory, more case studies. Trainers should include more technical and practical details in their course materials, as the surveyed participants found that much of the material was weak in this respect.

  • More trainings on specific issues instead of focusing on general and basic topics. Discussions within the EPA have also been moving in this direction.

  • Self-sustainability. Developing strategies for more self-sustainability is necessary as funding will decrease in the years to come. Some examples of ways to raise funds include workshop fees and the sale of expertise or of database material.

The complete report can be obtained from the Institute for Sustainable Communities, 56 College St., Montpelier, Vermont 05602-3115, U.S.A.; Fax: (802) 229-2919.


REC * EMTC * PUBLICATIONS * INSIGHT * AUTUMN 1997

PREVIOUS NEXT COVER PAGE HOME PAGE