Push Notifications
provide the capability to send information based on subscribed content to devices. The
push-notifications
service accepts notification requests via HTTP for batching and submission to native application push provider APIs. It is deployed in
Kubernetes
, and its request intake API endpoints are exposed only internally to the cluster. In the future it might be updated to handle web push (see
phab:T266138
).
Lifecycle of a push notification request
Push notifications build on the longstanding MediaWiki notification system implemented in the
Echo
extension. It adds a third notification type, currently labeled "Apps," to the preexisting "Web" and "Email" types. It is the responsibility of the app clients to register users' push subscriptions via the
echopushsubscriptions
API module and update their notification preferences in order to benefit from push notifications.
When an Echo event occurs and a recipient's notification preferences indicate that the user has chosen to be notified of that event type by push, Echo pushes a
NotificationRequestJob
onto the job queue. Upon execution of the job, the database is queried for all of the user's registered push subscriptions, and requests are made to the
push-notifications
service (to endpoints
/v1/events/fcm
and
/v1/events/apns
, for Firebase Cloud Messaging (Android) and Apple Push Notification Service (iOS), respectively) to request that a message be sent to each subscribed device.
Upon receiving a push notification request, the push-notification service enqueues the request to be batched with other requests for a configured period of time. This period can be either defined explicitly or configured to be randomized between a provided pair of lower and upper bounds. After the configured batching period passes, the request is consolidated with other requests received during the batching period into one or more multicast requests, and submitted to the push provider API. Consolidation with other messages into generic multicast requests is possible because the system currently sends only a generic message prompting the apps to contact the MediaWiki API to receive full message content.
[TO BE IMPLEMENTED: see
T260247
and subtasks] When a push provider API responds that a subscriber token is invalid, the
push-notifications
service submits a request to the MediaWiki API to delete that subscription from the database.
Dashboards
The service is sending metrics to Prometheus via service-runner.
Job queue processing metrics for NotificationRequestJobs can also be seen on the JobQueue EventBus dashboard:
Connecting to push-notifications
Production
The service is exposed by service proxy at
To access the service directly, by cluster:
To quickly verify that the service is up and running, request
/_info
mholloway-shell@deploy1001:~$ curl -s http://localhost:6012/_info | jq .
"name": "push-notifications",
"version": "0.0.1",
"description": "A service for sending push notification messages",
"home": "https://www.mediawiki.org/wiki/Wikimedia_Product_Infrastructure_team/Push_Notifications_Infrastructure"
Beta Cluster
role::beta::docker_services
instance of the service is running on
deployment-push-notifications02
and exposed internally at
Deployment process
Production
See
Deployments_on_kubernetes
Beta Cluster
In
Horizon
, navigate to the
deployment-prep
(Beta Cluster) project and view the
deployment-push-notifications
tab under "Prefix Puppet". You will see a configuration definition like the following:
profile::docker::engine::declare_service: true
profile::docker::engine::packagename: docker.io
profile::docker::engine::settings: {}
profile::docker::runner::service_defs:
mediawiki-services-push-notifications:
config:
logging:
level: trace
metrics:
port: 9100
type: prometheus
num_workers: 0
services:
[...]
environment:
DEBUG: apn
namespace: wikimedia
override_cmd: -c /etc/mediawiki-services-push-notifications/config.yaml
port: 8900
version: 2021-08-20-154251-publish
Update the
version
setting with the desired image tag from the pipeline. The update will be picked up and the service will be restarted with the new image on the next Puppet run.
In case of emergency
Push notifications support can be disabled without negative effects on other wiki operations. [TODO: Verify job queue retry behavior and ensure there are no spillover effects]
To disable push notifications support in MediaWiki (Echo), set
$wgEchoEnablePush
to
false
. This will result in the push subscription management API module being removed, and push notification request jobs no longer being sent to the job queue for submission to the
push-notifications
service.
Service level objectives (SLOs)
TODO:
T264118
See also