Grants:APG/Proposals/2017-2018 round 1/Wikimedia Israel/Staff proposal assessment

The staff proposal assessment is one of the inputs into the Funds Dissemination Committee (FDC) proposal review process, and is reviewed by the FDC in preparation for and during the in-person deliberations each round. The purpose of the staff proposal assessment is to offer the FDC an overview of the expert opinions of the FDC staff team about this annual plan grant proposal, and includes (1) A narrative assessment; (2) An assessment that scores each applicant according to specific criteria in program design, organizational effectiveness, and budgeting.

Overview

Summary

	Current (projected)	Upcoming (proposed)	Proposed change (as a +/- percentage)
FDC or other relevant funding	$239,519	$299,398	25%
Budget	$375,181	$457,932	22.06%
Staff (FTE)	5.1	5.4	5.8%

Overview

This section summarizes the themes that emerged from the proposal, which are explained in the detailed narrative section of this assessment.

Themes

• Opportunities for GLAM
• Thoughtful approach to evaluation
• Convoluted plan
• Salary adjustments make sense

Wikimedia Israel requires a substantial increase in their grant amount this year, with the rationale that they need to adjust their salary range to market prices. This increase may be long due, and shows both frugality and maybe a lack of anticipation on their part in their year to year growth. Their 2018 plan focuses on past successful programs but is a bit more convoluted than we are used to. WMIL is expanding its programmatic scope this year with a deep dive into the GLAM sector, and continues to build upon flagship programs, such as their education programs, especially the Students edit Wikipedia in Arabic program. WMIL continues to have an excellent approach to learning that informs the way it develops its programs. We are looking forward to seeing WMIL's next iteration of their strategic plan.

Staff proposal assessment narrative

This section takes an in-depth look at this organization's past performance and current plans.

Context and effectiveness

This section takes a close look at this organization's context. Here are some questions we will consider:

Environment

How does this organization's environment (including its local context and its community) enable this plan to succeed and have impact?

WMIL has the opportunity to host the GLAM Wiki Conference in Israel in 2018, which should help them improve relationship with GLAM partners in Israel, and will be an important chance for them to contribute to the international movement. While their experience in GLAM is still quite limited, they have had ongoing partnerships and have supported small GLAM operations. We see this as an opportunity to explore nascent relationships in that domain, and trust WMIL to do this right.

Wikimedia Israel has been working hard at developing their Students edit Wikipedia in Arabic program, in a constant care to foster diversity. This is a challenging but very rewarding program for which they have also been able to secure external funding, and are seeing good results so far. In past assessments we have commented on this important opportunity, and we are glad to see this work progressing.

Past performance

Will this organization's past performance with program implementation enable this plan to succeed?

WMIL has a history of achieving good results in some areas, such as participants involved and qualitative achievements in the education sector.

For example, at the 2017 midpoint WMIL reports that a Wikipedia article writing assignment was included in the matriculation exam in history for the first time. This is an important achievement that can lead to significant results in the long term. Their involvement with the Ministry of Education is also an assurance of the sustainability and scalability of their programs.

WMIL’s achievements in numbers of participants are impressive, with more than 3,500 participants at the midpoint, and they have also set ambitious yet achievable targets in this area for 2018. Achievements for pages are at about 13,000 at the midpoint.

Organizational effectiveness

Will this organization's overall effectiveness enable this plan to succeed?

Wikimedia Israel has been working for many years to establish a sound organizational structure. While there have been bumps along the way, WMIL has achieved a lot and has built the capacity they need to be effective in the areas they are prioritizing.

This past year was particularly difficult, as many staff members departed the organization and WMIL invested significant resources in training and retraining new staff. We appreciate that WMIL is taking action to address the risk of turnover by increasing staff salaries. This adjustment needed, as WMIL now needs to bring on and retain program staff with specific expertise in order to be effective.

Strategy and programs

This section takes a close look at this organization's programs. Here are some questions we will consider:

Strategy

Does this organization have a high-quality strategic plan in place, and are programs well-aligned with this strategic plan?

WMIL's strategic plan was developed two years ago for two years. It has informed WMIL's development of programs. WMIL took an active part in the Wikimedia strategic process and is planning to develop their own strategic plan in 2018, based on the Wikimedia strategic directions and their implementation.

The existing strategic goals (nurturing and diversifying open-content communities, making knowledge and information open and accessible for all and raising public awareness to the importance of open and data and open knowledge) seem to be present in all programs but are not clearly referenced.. Given the large number of programs included in this proposal, this makes it very difficult to follow exactly how the strategy is being implemented. This can be improved in the next iteration of their annual plan.

Programs

Do proposed programs have specific, measurable, time-bound objectives with clear targets, and are program activities and results logically linked? Are there specific programs with a high potential for impact?

Wikimedia Israel's program structure is convoluted. For example, their education program appears to be a group of several programs that have distinct goals and targets. It is difficult to understand how these three programs share common goals or are linked as part of a system. Based on what we know of WMIL’s past work and approaches, we think it is likely this logic exists. Yet, it is not clearly articulated here. We believe this structure can be improved in subsequent proposals, but we still appreciate that WMIL’s programs have a high potential for impact. In particular, their education work has the potential to scale at the national level, and WMIL continues to engage their community while expanding to include more communities through education work in the Arabic language. WMIL has given due attention to properly resourcing this area (Arabic), and we hope that this pays off with good results in the upcoming year.

With respect to their evaluation plan, we would like to express appreciation as well as concern.

We see that WMIL is carefully considering and choosing metrics that are both nuanced and relevant for each of their many programs. We appreciate that WMIL values the context of each program and is seeking to provide the most relevant means possible for expressing program outcomes by assigning individual grantee-defined metrics at the program level. Their metrics emphasize challenging and relevant outcomes such as sustainability and retention, and this information is likely to help WMIL in assessing and improving their work. However, we have concerns that the end result will be difficult to interpret, especially considering the large number of programs included in this proposal overall.

This nuanced approach is especially valuable since some of WMIL’s high level targets are low relative to what other organizations are planning and achieving (pages, for example). We appreciate that targets for participation and new editors are very competitive.

We also appreciate WMIL’s willingness to adapt and change when certain metrics are not working for them. For example, they discarded the quality metric that was not practical to implement on the scale they had planned.

Finally, we ask that WMIL not include communications as a program in future proposals, as this in an area that is rarely programmatic. We agree it is valuable for the organization to develop a communications strategy and track their progress in this important area, but this should not be included at the same level as programs.

Budget

Is this plan and budget focused on the programs with the highest potential for online impact?

WMIL's rationale for increasing their budget is an alignment of salaries on sector prices, which makes sense in their current organizational context. WMIL's staff is mostly programmatic, which means that while salary increases will mostly be allocated towards operations through the executive's salary increase, the budget for operations stay at a healthy level for an organization this size.

Summary of expert opinions (if applicable)

This section will summarize any expert opinions or other research.

N/A

Staff proposal assessment framework

This framework assesses annual plan grant proposals across the three dimensions of (1) Program design, (2) Organizational effectiveness, and (3) Budgeting. To complete the assessment, we identify whether each criterion is a strength or a concern:

• Major strength
• Strength
• Neither a strength nor a concern
• Concern
• Major concern

Criterion	Assessment	Description
Program design
P1. Strategy	Neither	WMIL's strategic plan was designed for 2 years and while it does not perfectly align with WMIL's programs, it provides a good backbone to them. WMIL has participated in the Wikimedia strategy process and intends to review its strategic planning in 2018 to integrate the results of the Wikimedia strategy process and align with the strategic direction.
P2. Potential for impact at scale	Strength	Despite less clarity as previously in their plan, WMIL's programs have the potential to scale, especially in education. The success of the Arabic program shows WMIL's commitment to outreach across the country and population. Foraging into GLAM at an international level through the GLAM conference could bring impact at scale in other programs as well.
P3. Objectives and evaluation methods	Neither	Wikimedia Israel has developed very specific metrics for each to its programs. While this granularity may strengthen the scalability and adaptation of their program and activities, this very complex evaluation plan will be a challenge to implement.
P4. Diversity	Strength	WMIL has steadily continued to deliberately strengthen its outreach to diverse populations, with the continuous support of Wikiwomen, Wiki developers, but also through its Arabic program.
Organizational effectiveness
O1. Past results	Strength	WMIL's results in 2016 are exceeding expectations, except in the realm of new editors. At the midpoint, WMIL seems to be on track to exceed ts targets for 2017. They have also had some important qualitative achievements, WMIL reports that a Wikipedia article writing assignment was included in the matriculation exam in history for the first time.
O2. Learning	Major Strength	WMIL continues to have a strong culture of learning. Their impact and progress reports especially show a thorough analysis of their programs and how these findings inform the way they adapt.
O3. Improving movement practices	Strength	WMIL is planning to host a GLAM conference this year, putting at the service of the movement its skills in event organizing. WMIL works with other groups in the movement and has participated in the development of the strategy direction.
O4. Community engagement	Strength	WMIL still works closely with its community, and has managed to secure continuous support for meet ups and activities. The hire of a community manager issued from the Hebrew Wikimedia community has allowed WMIL to strengthen their capacity for listening and supporting.
O5. Capacity	Neither	WMIL has gone through a difficult period where many staff members left the organization. This has impacted their resources by having to train staff over and over again. As they reach more stability, it is good to see that they have strategies for addressing the risk of staff turnover in the long term.
Budget
B1. Past budgeting and spending	Neither	While WMIL has managed to break even in the long run, it has a history of stretching its resources, which hopefully is being addressed this year and will help plan more realistically. WMIL has been successful in securing recurring funding from external donors.
B2. Budget is focused on impact	Strength	WMIL 's budget is geared toward programmatic impact, as most of its staff is program staff. Despite a lot of the budget increase ask going towards operations (executive), WMIL is still a lean organization.

This staff proposal assessment is the work of FDC staff and is submitted by: Morgan Jue (WMF) (talk) 01:32, 9 November 2017 (UTC)

Staff proposal assessment framework

• Major strength. This is something the organization does very well, and this is a strong indicator of future success.
• Strength. This is something that the organization does well, and this could indicate future success.
• Neither a strength nor a concern. This is something that does not indicate future success or make funding the application a risk, or aspects of this criterion conflict.
• Concern. This is something that the organization does not do well, and this could make funding the application a risk.
• Major concern. This is an area where the organization is not strong, and this could make funding the application a serious risk.

Criterion	Description
Program design
P1. Strategy	The organization has a quality strategic plan in place, programs are aligned with this strategy, and this strategy is aligned with online impact.
P2. Potential for impact at scale	Programs could lead to significant online impact at scale, and corresponding to the amount of funds requested
P3. Evaluation methods	Programs include a plan for measuring results and ensuring learning, and employ effective evaluation tools and systems. Programs include SMART objectives, targets, and logic models.
P4. Diversity	Programs will expand the participation in and reach of the Wikimedia movement, especially in parts of the world or among groups that are not currently well-served.
Organizational effectiveness
O1. Past results	This organization has had success with similar programs or approaches in the past, and has effectively measured and documented the results of its past work.
O2. Learning	This organization is addressing risks and challenges effectively, is learning from and documenting its experiences, and is applying learning to improve its programs.
O3. Improving movement practices	This organization effectively shares learning about its work with the broader movement and beyond, and helps others in the movement achieve more impact.
O4. Community engagement	This organization effectively engages communities and volunteers in the planning and implementation of its work.
O5. Capacity	This organization has the resources and ability (for example, leadership, expertise, staff, experience managing funds) to do the plan proposed.
Budget
B1. Past budgeting and spending	This organization has a history of budgeting realistically and managing funds effectively in the past.
B2. Budget is focused on programmatic impact	Based on past performance and current plans, funds are allocated to programs and activities with corresponding potential for programmatic impact.