TL;DR: MediaWiki and extensions are deployed to all Wikipedias on a train every Thursday. There may be some advantages in moving deployment to some languages to Wednesday. This is an examination of the pros and cons of such move. Actions for particular languages should probably be discussed in separate tickets.
In the current schedule of the weekly deployment train, whatever is merged before the California morning every Tuesday is branched and deployed to mediawiki.org and several testing sites, then on Wednesday it's deployed to non-Wikipedia sites and on Thursday - to Wikipedia sites.
This generally works well, but it has a disadvantage for features that are only deployed on Wikipedia, such as ContentTranslation. Whereas features such as VisualEditor and MobileFrontend, not to mention the core, can be tested on mediawiki.org and other sites before they are live on Thursday and urgent issues can be fixed. testwiki can work, too, but it's not a perfect replication of the live Wikipedias - it doesn't have all of Wikipedias' articles, templates, user load and multilingualism.
I therefore suggest moving Wikipedias in some languages to Wednesday. The the current schedule page says that theoretically it's possible and has already been discussed in relation to the Chinese Wikipedia, but this hasn't actually happened yet. I spoke to @Kippelboy from the Catalan Wikipedia and to @KuboF from the Esperanto Wikipedia. @Kippelboy already started a discussion about this in the Catalan Wikipedia, and the community responses are generally positive - the editors community is willing to try it. (Request for Comment on Esperanto Wikipedia opened; -KuboF)
So, using Catalan just as an example for now, here's how it is supposed to work:
- The new branch is created on Tuesday and deployed to mediawiki.org and test* sites.
- It is deployed to non-Wikipedia sites and to the Catalan Wikipedia on Wednesday.
- If anybody notices very severe ("Unbreak Now") bugs on non-Wikipedia sites or on the Catalan Wikipedia after the train runs, these bugs are fixed and backported to the branch.
- The branch is deployed to all Wikipedia sites on Thursday; the urgent fixes, if there are any, are deployed to all languages, including Catalan.
- If this doesn't work well for the community for whatever reason, we revert to the current way - all Wikipedias on Thursday.
Advantages:
- Ability to test Wikipedia-specific features early in a real Wikipedia environment, with real content, gadgets, templates, and users.
- Ability to fix urgent bugs before they hit the other languages on Thursday.
- In particular, this means that it's possible to fix urgent stuff before the weekend. Usually deployments aren't done on Friday, except the super-urgent ones. So all languages, including those that are deployed early, get the chance to enjoy the fixes earlier.
- No special SWAT deployments should be needed to make this possible. This is supposed to become a part of the deployment train configuration.
- Rolling deployments are a generally good industry practice.
Possible concerns:
- This must be compatible with the deployed services, like Parsoid and cxserver.
- Care must be taken that ContentTranslation, and possibly other features, is compatible across languages. For example, ContentTranslation uses a common database to store translations in progress, suggestion lists, etc., and the database usage must be in sync across all the projects. Usually, this is not supposed to be a blocker, but it may require a bit more diligence in deployment. This may also apply to some other features.
- Catalan is not necessarily the best first language for testing. It is an advantage to test with a lot of users, but it's a problem if something breaks, so it may be worth picking some languages with less activity for the testing.
- There's little time to translate new messages that come with Tuesday's latest patches (but it's not new because it affects non-Wikipedias already, and it is not much of problem if the translators are quick).
Of course, there may be more points in both lists. Your further opinions and analysis about pros and cons are welcome.
The actual configuration work to make this happen will probably have to be done by Release Engineering and Ops, but there are a lot of stakeholders, hence the long CC list.
Thanks.