Grants:APG/Proposals/2017-2018 round 1/Wikimedia UK/Staff proposal assessment

The staff proposal assessment is one of the inputs into the Funds Dissemination Committee (FDC) proposal review process, and is reviewed by the FDC in preparation for and during the in-person deliberations each round. The purpose of the staff proposal assessment is to offer the FDC an overview of the expert opinions of the FDC staff team about this annual plan grant proposal, and includes (1) A narrative assessment; (2) An assessment that scores each applicant according to specific criteria in program design, organizational effectiveness, and budgeting.

Overview

Summary

	Current (projected)	Upcoming (proposed)	Proposed change (as a +/- percentage)
FDC or other relevant funding	$383,885	$432,274	12.61%
Budget	$855,838	$970,091	13.35%
Staff (FTE)	9	9.6	6.66%

Overview

This section summarizes the themes that emerged from the proposal, which are explained in the detailed narrative section of this assessment.

Themes

• Impressive achievements in 2017
• Emphasis to diversity
• Potential for movement contributions

WMUK’s 2017 achievements are impressive. They have created a high quality plan for 2018 that is aligned with their strategy and includes an emphasis on diversity, including relevant metrics. They are continuing to emphasize volunteer involvement, and are achieving good results there, and are developing their sophisticated approach to partnerships across their programs.

Education program work seems less likely to lead to impact on the Wikimedia projects, and WMUK could be more selective in the top-level objectives it chooses to feature for each program. WMUK is an organization with strong capacity, and they have done work in this area over the past several years. Now that the organization has this capacity and is achieving results, we want to encourage WMUK to consider developing their role in the international movement.

Staff proposal assessment narrative

This section takes an in-depth look at this organization's past performance and current plans.

Context and effectiveness

This section takes a close look at this organization's context. Here are some questions we will consider:

Environment

How does this organization's environment (including its local context and its community) enable this plan to succeed and have impact?

WMUK has some important opportunities in their environment, as we have emphasized in previous assessments.

In the past we have identified WMUK’s opportunities for engagement with non-English communities. We now see that they have taken good advantage of these, and will continue this emphasis in next year’s plan. We have also pointed out that working with English Wikipedia is both a challenge and an opportunity, and this continues to be the case.

The richness of partnership opportunities in the UK also continues to be emphasized in the current year and in next year’s plan.

Past performance

Will this organization's past performance with program implementation enable this plan to succeed?

Last year we expressed appreciation for WMUK’s improved results in 2016, and we continue to see steady improvements in 2017.

For example, WMUK has been measuring volunteer hours as an indication of how involved volunteers are with their programs, and they achieved a total of 10,116 volunteer hours by the midpoint of 2017. This puts them on track to continue to build on this achievement, which is very important to their internal understanding of how well they are working.

WMUK has achieved almost 250,000 pages added or improved by the midpoint of 2017, which is a remarkable achievement. These do include a large portion of Wikidata edits (around 166,000) and at least one large content donation of 12,000 audio files. Still, we know from some breakdowns of metrics in other sections of the progress report, that very significant numbers of articles are being generated through GLAM activities. WMUK is also achieving articles on projects other than English Wikipedia, which is an important aspect of considering their work on content diversity.

WMUK has achieved success with their Welsh language focus in the past, and also included work with the Gaelic Wikipedia community in 2017. In 2017, WMUK helped to establish a permanent Wikimedian at National Library of Wales, which represents an achievement in their target area of facilitating cultural change within key institutions and has also led to large amounts of content being contributed. The bilingual focus of this residency is an especially important aspect of this work.

In terms of qualitative achievements, WMUK supported the first Wikimedia conference for minority languages in 2017, in partnership with Edinburgh University. This event was international in scope and included more than 50 participants. It integrates nicely with the work WMUK is doing on minority languages with the UK, and is also important in that it shows they are taking on a leadership role in this area internationally.

Compared with results from these other programs, achievements in the area of education (classroom program) are very modest, at just 1,000 articles achieved at the midpoint. This program area also include residents that are embedded at educational institutions however, which is a model that WMUK has had success with.

Organizational effectiveness

Will this organization's overall effectiveness enable this plan to succeed?

After a period of turmoil several years ago, Wikimedia UK appears to have stabilized their staffing structure,. With quite a large staff in the Wikimedia movement, they seem to be allocating staff resources to program areas with good potential for impact. The request for a person to focus on Scottish activities is a good example of this, as this appears to be an area with high potential that could benefit from more paid staff support.

WMUK has included a detailed explanation of their process for doing needs assessments that help them determine whether or not to pursue new projects or activities, and we enjoyed learning more about this model. We appreciate how far WMUK has come in the past several years in terms of their expertise in prioritizing and explaining the relevance of their different program activities.

WMUK has also come a long way in emphasizing their partnership with volunteers, and in explaining how volunteers are involved in every aspect of their work. We appreciated the detailed context given about the way they consult the community about organizational priorities, and also the tracking of volunteer involvement across their various initiatives. In the long term, maintaining or increasing volunteer involvement will be an important aspect of WMUK’s capacity, and so it is heartening to know they are prioritizing and tracking this aspect of their work.

In previous year’s we have commented on the success of WMUK’s efforts to revitalize their board. We feel it is important to re-emphasize this here, as it is clear from their 2017 achievements and their 2018 plan that these efforts are bearing fruit.

Strategy and programs

This section takes a close look at this organization's programs. Here are some questions we will consider:

Strategy

Does this organization have a high-quality strategic plan in place, and are programs well-aligned with this strategic plan?

Last year, we commented on WMUK’s new strategic plan and were expectant as to how it would be executed and we have not been disappointed. Their 2018 annual plan is also well-aligned with the strategic plan so we can expect even better results, as its implementation benefits from WMUK's experience.

Programs

Do proposed programs have specific, measurable, time-bound objectives with clear targets, and are program activities and results logically linked? Are there specific programs with a high potential for impact?

In the past we have celebrated WMUK’s sophisticated approach to partnerships, which is an aspect of each of their programs in this year’s proposal. We are glad to see that they are continuing to refine and develop this approach in this year’s proposal, and believe this approach can lead to impact in 2018.

WMUK has presented a concise plan that is effective in explaining what they are doing and why, and it also includes sets of targets and objectives for each program that have different degrees of relevance. For example, while we are glad to see WMUK measuring volunteer hours for each program as a way of understanding and articulating how volunteers are involved in this work, this may not be relevant enough to each program’s high level goals to include as a program objective. For each program, 2-3 high level objectives could be included to emphasize the change WMUK is trying to achieve. This would strengthen the focus of each program.

WMUK’s logic models for each program have been helpful to understand the way they see each program working; however, even in these logic models there is not always enough distinction between more or less important goals. For example, in the advocacy logic model two outcomes are listed: (1) Wikimedia has been recognized as a leading organization for open knowledge and; (2) Our work has significantly increased access to knowledge about or held in the UK. In our opinion, these two objectives are not on the same level. It seems that the second objective captures what Wikimedia UK ultimately wishes to achieve through this program, while the first should be a means to achieve that outcome and not an end in and of itself. In this case, focusing on the highest level outcome would have been more effective.

The program Diverse content and contributors is a high potential program. We consider this, however, to be two separate programs, with distinct theories of change and different relevant objectives. If this is to be considered a unified program, more explanation should be offered about how these logic models are interrelated. Despite this structural issue, we agree this is a high potential program. For example, work on Celtic languages is particularly interesting, as is the focus on immigrant languages. This really takes WMUK’s diversity work to the next level, showing that they are taking good advantage of the language diversity present in their working environment, and not focusing only on English Wikipedia. A focus on organizational partnerships to connect with diverse communities is likely to continue to lead to good results in these areas, since partners can bring some much-needed expertise to this work.

Helping learners in the UK become able to understand and engage with open knowledge is a noble goal, and it is less common to see such goals linked with education programs. On another level, the target for the education program is rather modest at 1,800 articles improved, and so this program may have limited direct impact on the projects in the long term, especially when compared with WMUK’s other two programs. It will be interesting to see how these two outcomes inform developments of this program.

Budget

Is this plan and budget focused on the programs with the highest potential for online impact?

Results have been very good, and staff resources are generally well-directed. We are glad WMUK is supporting a strong capacity in the area of finance and administration and is planning to use their fundraising expertise, both for gifts in kind and other fundraising aspects. Staff costs are weighted toward programs, and in general WMUK’s programs are leading to results which provide a good foundation for further support. At this level of funding, it would be interesting to see how WMUK can contribute in a more global fashion, by focusing resources towards global programs or activities.

Summary of expert opinions (if applicable)

This section will summarize any expert opinions or other research.

N/A

Staff proposal assessment framework

This framework assesses annual plan grant proposals across the three dimensions of (1) Program design, (2) Organizational effectiveness, and (3) Budgeting. To complete the assessment, we identify whether each criterion is a strength or a concern:

• Major strength
• Strength
• Neither a strength nor a concern
• Concern
• Major concern

Criterion	Assessment	Description
Program design
P1. Strategy	Major strength	WMUK has fulfilled the expectations set in our previous year's assessment. Their excellent strategic plan is leading to improved results and continues to be a strong backbone to their programs.
P2. Potential for impact at scale	Strength	WMUK has proven that their results scale year after year. Despite a budget decrease last year, they have managed to attain their targets. Work in Scotland is a good example of scaling programs and adapting them to succeed in different environments.
P3. Objectives and evaluation methods	Strength	WMUK's plan includes SMART objectives, relevant metrics, and clear targets. Their volunteer hours is an interesting metric. Despite this, not all objectives presented seem equally relevant to the main goal of each program, and this can be improved in future years.
P4. Diversity	Major strength	WMUK's plan address several aspects of diversity relevant in the UK context and is including ways to measure progress in these areas. Furthermore, we find their approach to partnering with groups of relevant expertise particularly likely to be effective. In this plan, WMUK is extending their impact far beyond English Wikipedia, and is putting real resources behind these initiatives.
Organizational effectiveness
O1. Past results	Strength	WMUK's work continues to generate impressive results in terms of content created or improved (although they are challenging to disaggregate in a systematic way). They are doing very well against their own goal of volunteer hours, achieving more than 10,000 volunteer hours by the midpoint of 2017. Important qualitative achievements include the launch of the Celtic Knot conference in 2017.
O2. Learning	Strength	WMUK operates on an iterative framework to assess its performance, risks, financial impact and needs. It constantly corrects course when needed and adapts its activities and programs on a regular basis, providing the necessary rationale.
O3. Improving movement practices	Strength	WMUK makes room for close collaboration with other chapters, participating and giving their input through training in movement conferences, and researching paths of collaboration with other affiliates. They are also contributing to FKAGEU.
O4. Community engagement	Strength	Wikimedia UK for the past two years has been focusing on volunteer engagement. They have made "volunteer hours" one of their flagship metric and put a lot of emphasis on volunteer leadership. They have managed to build bridges and engage with various communities, both in the English language space and in minority languages.
O5. Capacity	Major Strength	WMUK has achieved several important things in recent years that strongly indicate they are an organization with the capacity to deliver on their plans. For example, we are impressed with the work they did to revitalize and develop their board. Beyond this, they have emerged from a period of difficult staff transitions towards a period of stability. They have a team with relevant expertise and experience, and ways of working together. Finally, their emphasis on volunteer involvement will likely improve the organization's capacity in the long term.
Budget
B1. Past budgeting and spending	Strength	WMUK has delivered on results in the past year and has shown reactivity and caution in their budget reallocation. Impact is now more commensurate with funding.
B2. Budget is focused on impact	Strength	While this is a large amount of funding, it seems likely that WMUK can deliver on a plan that will lead to impact. We see a wise allocation of staffing resources, to areas with growing potential such as the Scottish coordination work.

This staff proposal assessment is the work of FDC staff and is submitted by: Morgan Jue (WMF) (talk) 01:38, 9 November 2017 (UTC)

Staff proposal assessment framework

• Major strength. This is something the organization does very well, and this is a strong indicator of future success.
• Strength. This is something that the organization does well, and this could indicate future success.
• Neither a strength nor a concern. This is something that does not indicate future success or make funding the application a risk, or aspects of this criterion conflict.
• Concern. This is something that the organization does not do well, and this could make funding the application a risk.
• Major concern. This is an area where the organization is not strong, and this could make funding the application a serious risk.

Criterion	Description
Program design
P1. Strategy	The organization has a quality strategic plan in place, programs are aligned with this strategy, and this strategy is aligned with online impact.
P2. Potential for impact at scale	Programs could lead to significant online impact at scale, and corresponding to the amount of funds requested
P3. Evaluation methods	Programs include a plan for measuring results and ensuring learning, and employ effective evaluation tools and systems. Programs include SMART objectives, targets, and logic models.
P4. Diversity	Programs will expand the participation in and reach of the Wikimedia movement, especially in parts of the world or among groups that are not currently well-served.
Organizational effectiveness
O1. Past results	This organization has had success with similar programs or approaches in the past, and has effectively measured and documented the results of its past work.
O2. Learning	This organization is addressing risks and challenges effectively, is learning from and documenting its experiences, and is applying learning to improve its programs.
O3. Improving movement practices	This organization effectively shares learning about its work with the broader movement and beyond, and helps others in the movement achieve more impact.
O4. Community engagement	This organization effectively engages communities and volunteers in the planning and implementation of its work.
O5. Capacity	This organization has the resources and ability (for example, leadership, expertise, staff, experience managing funds) to do the plan proposed.
Budget
B1. Past budgeting and spending	This organization has a history of budgeting realistically and managing funds effectively in the past.
B2. Budget is focused on programmatic impact	Based on past performance and current plans, funds are allocated to programs and activities with corresponding potential for programmatic impact.