Chinese

Terms of Reference for the Mid-Term Evaluation 

 UNDP/GEF Project

Demonstration for Fuel-Cell Bus Commercialization in China (Phase II)

In response to the global environment challenges linked to the air pollution problems and the public transportation needs in the developing countries, the project on Demonstration for Fuel Cell Bus (FCB) Commercialization (Phase II) was supported by the Global Environment Facility (GEF) and UNDP, and implemented by the Ministry of Science and Technology (MoST). The FCB II Project covers Beijing and Shanghai, two mega-cities, which have different geography, climate, road infrastructures, traffic conditions and severity, as well as social and market conditions. Both mega-cities are facing fast growing needs to upgrade the public transit and to address the critical challenge of air pollution.

The Project Document for the China FCB Project II was approved by GEF in July 2006 and the Project Document was signed in May 2007. The project was launched in November 2007.The goal of the project is to reduce GHG emissions and air pollution through widespread commercial introduction of FCBs in urban areas of China. The objective of this project is to demonstrate the operational viability of FCBs and their refueling infrastructure under Chinese conditions.

Partnership building between the private sector and the government is crucial for the success of the project inasmuch as the commercialization of FCBs, which are new environmentally-friendly means of transportation in China, will involve the active and serious participation of the private sector. The project involves the development of strategies to build and expand such partnerships. As part of the project, the procurement of the six FCBs from Shanghai Automotive Industry Corporation (SAIC) and the construction of the hydrogen refueling station in partnership with Shell and Chinese partners have resulted from such kind partnership. These partnerships not only enable the business sector to enter this potential market but also lower the cost to be covered by the government to improve the public services of transport and environment.

2. Description of the Assignment

As part of its project management activities, the China FCB II project is up for mid-term review (MTR). The purpose of the MTR is to evaluate the project implementation and management performances. It will determine whether the project is on track to achieve the project objective and therefore just need to be sustained; or needs additional guidance/assistance to keep it on track. From the evaluation findings, the MTR will come up with recommendations that will help guide the project implementation from this point in the project’s time line (i.e., more or less midway of the project implementation period) till end of the project to achieve its objective. The project evaluation will also determine and report on the experiences and lessons learnt during the project implementation so as to provide guidance in determining the targets and strategies for the planned next Phase of the China FCB Commercialization Project. In this regard, the findings and recommendations of the evaluation will contribute to identify best possible ways to advance the modern public transportation technology by commercializing FCBs in the cities.

 3. Scope of the Evaluation

The scope of the MTR covers the entire UNDP/GEF-funded FCB Phase II project and its components as well as the co-financed activities in each component of the project.

The MTR will assess the FCB II Project implementation taking into account the status of the project activities and outputs and the resource disbursements made up to 31 October 2010.

The evaluation will involve analysis at two levels: component level and project level. On the component level, the following shall be assessed:

·        Whether there is effective relationship and communication between/among components so that data, information, lessons learned, best practices and outputs are shared efficiently, including cross-cutting issues;

·        Whether the performance measurement indicators and targets used in the project monitoring system are specific, measurable, achievable, reasonable and time-bounded to achieve desired project outcomes; and,

·        Whether the use of consultants has been successful in achieving component outputs.

The MTR will include such aspects as appropriateness and relevance of work plan, compliance with the work and financial plan with budget allocation, timeliness of disbursements, procurement, coordination among project team members and committees, and the UNDP country office support. Any issue or factor that has impeded or accelerated the implementation of the project or any of its components, including actions taken and resolutions made should be highlighted.

At the project level, the MTR will assess the project performance in terms of: (a.) Progress towards achievement of results, (b.) Factors affecting successful implementation and achievement of results, (c.) Project Management framework, and (d.) Strategic partnerships.

(a)   Progress towards achievement of results (internal and within project’s control)

·        Is the Project making satisfactory progress in achieving project outputs vis-à-vis the targets and related delivery of inputs and activities?

·        Are the direct partners and project consultants able to provide necessary inputs or achieve results?

·        Given the level of achievement of outputs and related inputs and activities to date, is the Project likely to achieve its purpose/objective and contribute to the realization of its goal?

·        Are there critical issues relating to achievement of project results that have been pending and need immediate attention in the next period of implementation? 

(b)   Factors affecting successful implementation and achievement of results (beyond the Project’s immediate control or project-design factors that influence outcomes and results)

·        Is the project implementation and achievement of results proceeding well and according to plan, or are there any outstanding issues, obstacles, bottlenecks, etc. on the consumer (riding public), government (national and local) or private sector (e.g., bus companies, FCB manufacturers) or the public transport sector as a whole that are affecting the successful implementation and achievement of project results?

·        To what extent does the broader policy environment remain conducive to achieving expected project results, including existing and planned legislations, rules, regulations, policy guidelines and government priorities?

·        Is the project logical framework and design still relevant in the light of the project experience to date?

·        To what extent do critical assumptions/risks in project design make true under present circumstances and on which the project success still hold? Has the project team validated these assumptions as presently viewed by the project management and determine whether there are new assumptions/risks that should be raised?

·        Is the project well-placed and integrated within the national government development strategies, such as community development, poverty reduction, etc., and related global development programs to which the project implementation should align?

·        Do the Project’s outcomes remain valid and relevant, or are there items or components in the project design that need to be reviewed and updated?

·        Are the Project’s institutional and implementation arrangements still relevant and helpful in the achievement of the Project’s objective and outcomes, or are there any institutional concerns that hinder the Project’s implementation and progress.

(c)   Project management (adaptive management framework)

·        Are the project management arrangements adequate and appropriate?

·        How effectively is the project managed at all levels? Is it results-based and innovative?

·        Do the project management systems, including progress reporting, administrative and financial systems and monitoring and evaluation system, operate as  effective management tools, aid in effective implementation and provide sufficient basis for evaluating performance and decision making?

·        Is technical assistance and support from project partners and stakeholders appropriate, adequate and timely?

·        Validate whether the risks originally identified in the project document and, currently in the APR/PIRs, are the most critical and the assessments and risk ratings placed are reasonable.

·        Describe additional risks identified during the evaluation, if any, and suggest risk ratings and possible risk management strategies to be adopted.

·        Assess the use of the project logical framework and work plans as management tools and in meeting with UNDP-GEF requirements in planning and reporting.

·        Assess the use of electronic information and communication technologies in the implementation and management of the project.

·        On the financial management side, assess the cost effectiveness of the interventions and note any irregularities.

·        How have the APR/PIR process helped in monitoring and evaluating the project implementation and achievement of results? 

(d)   Strategic partnerships (project positioning and leveraging)

·        Are the project partners and their other similar engagements in the FCB II project, strategically and optimally positioned and effectively leveraged to achieve maximum effect of the sustainable transport and Energy Efficiency program objectives for the country?

·        How do the project partners, stakeholders and co-financing institutions involved in the Project’s adaptive management framework?

·        Are there further opportunities for stronger collaboration and substantive partnerships identified to enhance the project’s achievement of results and outcomes?

·        Are the project information and progress of activities disseminated to project partners and stakeholders? Are there areas to improve in the collaboration and partnership mechanisms?   

4. Specific Tasks for the Evaluation

Through the review of pertinent documents related to the project such as project document, quarterly and annual progress reports, other activity/component specific deliverables, reports and evaluation, if there are any, etc; conduct of structured interview with knowledgeable parties (e.g., PMO, GOC personnel, Beijing & Shanghai government officials, Sub-Contracting Parties/Entities, National Consultants, UNDP Country Office Counterparts, members of the Project Steering/Advisory Committee/s, Project Beneficiaries or grantees, etc.); and visits to various pilot project sites the evaluation mission will carry out the following tasks:

1.      Review of the project design, and planning to find out whether: (a) the project approaches and strategy are sound; (b) the immediate objectives and outputs are properly stated and verifiable in the project logical framework; (c) the timeframe of the project is feasible and practicable; and, (d) Others.

2.       Review of project performance: timeliness and quality of inputs; timeliness and cost-effectiveness of activities undertaken; quality and quantity of outputs produced; achievement of outcomes; and a financial review against the project budget.

The project is now more or less halfway of its time line and as such progress should be measured against outputs stated in the project document. The evaluation will focus on such aspects as appropriateness and relevance of work plan, compliance with the work plan along side with budget allocation; timeliness of disbursements; procurement, quantity and quality of goods and services created; coordination among different project actors and UNDP country office support. Any issue that has impeded or advanced the implementation of the project or any of its components, including actions taken and resolutions made should be highlighted.

[Note: Whatever format is deemed appropriate for the presentation of the assessment results in the evaluation report, the evaluation should come up with a summary of information as in the following table.]

 

Activities		Budget
Planned	Actual	As per ProDoc	Actual Expenditures	% of Project Budget

 

3. Review the project impact: determine the extent to which the project objectives are expected to be achieved and what are the short-term and long-term impact of the project, including efficiency of the project, cost-effectiveness of the project;

 

4.      Study the government policies in development of FCBs for the Chinese cities and assess the relevance of the project against the national development priorities and objectives;

 

5.      Analyze the current FCB projects in other countries, and the domestic and international market developments in supply and demand of the technology to find out and advise potentials for possible project expansion;

 

6.      Analyze and report on the good practices and lessons learnt in partnership building of the project with companies and agencies in introducing the FCBs and designing and building the refuelling station in expanded demonstration cities in China

 

7.      Provide recommendations on the improvement or sustenance of the implementation of the remaining activities of the project; potential aspects of FCB commercialization that can be covered in a planned Phase II of the project; and actions to be taken to support the sustainable development of FC vehicle technologies and applications in China.

5. Qualifications

The MTR assignment requires an evaluation team that will consist of: one international expert and one national expert. Both experts should have basic knowledge of fuel cell technology development globally and practical experience in public-private partnership building for advancing the new technologies from the research to the market.

The experts should hold an advanced degree in studies related to the subject, and have at least 10 years of working experience in the area of energy and EE technologies (preferably in transport). The international expert will act as the team leader and the national expert will support in the gathering and provision of inputs on the national context in development of the technology, the public transport systems and the local FC vehicle development and production.The final evaluation report should be finzalized by the evaluation team.

It is desired that the international and national consultants have as many as possible the following qualifications:

1)  Project development, implementation and evaluation experience;

2)  Professional experience with clean energy vehicle R&D or management

3)  Knowledgeable about the relevant policies of the GEF;

4) Good communications and writing skills in English;

5)  Knowledge of GEF projects and project requirements;

6) Good experiences in working in China and with Chinese counterparts.

6. Roles and Responsibilities

The UNDP-GEF Regional Technical Advisor (based in Bangkok, Thailand) who oversees the UNDP-GEF FCB II project will assist the UNDP CO and the MTR team in preparing for the Mid-Term Review of the project. The MTR Team reports to the UNDP CO and MoST. The project’s executing agency (MoST) shall coordinate all relevant national and the international agencies and companies and provide in advance copies of the necessary documents needed by the evaluation expert/s. Likewise, the MoST shall arrange and finalize the itinerary/schedule for the MTR in consultation with all parties concerned. The project’s Chief Technical Advisor (CTA) will provide insights of the global FCB market to the MTR Team and discuss with them in detail the technology selection for future FCB development. The EA and UNDP CO will coordinate the logistical arrangements for the evaluation.

7. Support to the Evaluation Team:

UNDP will provide policy guidance to the MTR Team, and the PMO will arrange necessary briefings, background materials, meetings, travel and other logistical support.

The following documents and reports shall be provided to the MTR Team to assist them in the conduct of the mid-term evaluation:

a)       FCB Commercialization (Phase II) Project document (UNDP)

b)       Evaluation Guideline (UNDP)

c)        Project Implementation Report (UNDP)

d)       Comprehensive reports including subcontracts, technical reports, study tour reports, newsletters etc (PMO)

(c)   Strategic partnerships (project positioning and leveraging)

 8. MTR Schedule and Budget

The MTR is scheduled to be conducted in Oct 2010, for a period of 10 working days in China and 5 home base working days. A preliminary schedule for the MTR assignment in China is proposed as follows and shall be finalized by the PMO in consultation with respective agencies. The terms of payment for the services rendered by the consultants (evaluators) are based conformity on the UNDP standards.

Schedule in China

Day	Activity	Lead Agency
1	Briefing at UNDP office and review of the background materials	UNDP
2-3	Meetings with the project management office, the Beijing PMO and CICETE, Beijing Public Transportation Corporation etc.	UNDP
4-6	Visit to Shanghai to meet the Shanghai PMO  and  Shanghai Automitive Industry Corporation and hydrogen refilling station operation people	PMO
7-9	Drafting of the evaluation report	MTR Team
10	Debriefing to share recommendations and findings of the evaluation	MTR Team

 9. Outputs

The MTR Team is expected to deliver the following outputs:

1) An evaluation report presenting the mid-term evaluation results of the project and recommendations for the implementation of the remaining activities until end-of-project. The report should be submitted to UNDP CO and MoST before departure from Beijing. The documents should be submitted in electronic format.

2) Presentation of findings to UNDP CO and project key stakeholders in a wrap-up meeting in UNDP CO.

The findings of the evaluation will be used by Ministry of Finance as the GEF Operational Focal Point in China, MoST as the implementing partner and UNDP to better adjust project strategy and approaches to guide the project implementation in the remaining period.

10. Payment Schedule

50% of the total amount due the consultant will be paid upon signature of the contract for mobilization. The remaining 50% and the class ticket between hometown and Beijing, between Beijing  and Shanghai, is payable upon acceptance by UNDP of the evaluation report in its final form.

11. Start of the Assignment

The MTR shall commence on the second week of October.

12. Documents to be provided to the Consultants:

·        China FCB Commercialization (Phase II) Project document;

·        Project implementation reports (PIRs);

·        Comprehensive report of subcontracts; and

·        Other related documents 

 

Applications are to be sent to the following persons/emails:

Contact Persons: Yu Dan; Wang Ju

Tel: 010-63702975/ 63702973

Fax: 010-63702974

Email: yudan221@126.com    zzdwj@yahoo.com.cn

 

Application deadline: 14 October 2010