ArgoCD Notifications From Resource Actions

[blakeromano]

Summary

Currently the Notification Controller can send notifications based on changes that occur within an Application and the lifecycle of that application. The Notification Controller is unaware of things that occur outside the Application Object. We want to be able to send notifications for other things that may occur within an application that do not affect the Application Object, such as when a resource action is invoked.

Motivation

There is a desire to be able to audit and send notifications when a user initiates a resource action like restarting an Argo Rollout or Daemonset or Deployment.

Proposal

I think there's a few possible ways to do this which could be investigated....

Option One:

Currently when something like a Resource Action gets invoked the Argo Server will execute the Resource Action and then emit a Kubernetes Event. We could use the Kubernetes Event, update the Notification Service to listen to Kubernetes Events, and then read the Affected Object (the ArgoCD Application) and map that to a notification. This would also probably require updating the way these notifications are mapped in the ArgoCD configuration file.

Option Two:

We could look at making the Notification Service receive GRPC based requests, we could then have these be processed on the fly by the Notification Service and allow configuration of these notifications to be separate from the current notification configuration.

Option Three:

We could update Resource Actions so when they are invoked they actually write the details of the resource action run into the ArgoCD Application Status. We could format this something like:

status:
   resourceActions:
     invocations:
       - userInitiated: foo@foo.com
          resourceName: my-deployment
          resourceKind: Deployment
          apiVersion: v1
          namespace: default
          timeInvoked: 2024-05-15T03:33:07Z
          actionInvoked: restart

And we could ensure that we only keep for example the last 3 invocations of a resource action in the status of the application.

I think I lean towards option three as it also could solve an audibility issue of "I want to know what resource actions were taken" and we could at least provide some way to track that. This also would make it so all the current notification configuration should work out of the box with resource actions.