Is Modification of Listener Rules Necessary for Canary Releases in ECS?

[moko-poi]

Content:

Background:

In our current Canary Release approach for ECS, we can route traffic only to the DefaultListener. We are considering supporting the ability to route to specific listener rules.

Problem:

With the existing implementation, there are multiple listener rules linked to a single listener. This becomes a challenge when requests are distributed based on path patterns, as our current approach does not support this.

Points for Discussion:

Rationale for Current Implementation:

Why have we implemented traffic routing only for the default listener till now?
What are the limitations or considerations that led to this decision?

Proposed Solution - Adding getlistenerRuleArn:

By adding the getlistenerRuleArn function, we could potentially retrieve all rules associated with a listener.
Would this approach allow us to route Canary traffic based on specific listener rules?
What are the potential challenges or drawbacks of this implementation?

Feedback and Suggestions:

We welcome input from the community on this issue.
Are there alternative solutions or considerations we should be aware of?
How can we effectively manage multiple listener rules in the context of Canary Releases?

[t-kikuc]

Thank you so much for creating the discussion 🎉

I've just created the issue #4725 and attached the image to explain the use case as below:

Why is this needed:

• We need to specify the rules to modify other than default one.
  For example, one listener has multiple listener rules and they have different path conditions and target services.
• Now PipeCD updates only the Default rule.
  So even if we want to modify the rule-web below, only the Default rule is modified.

[t-kikuc]

And I created PR #4726, seeing your PR #4668 as reference, thank you.

This is my idea in the PR, written here #4726 (comment).
How about specifying the rules to modify in the input as below?

apiVersion: pipecd.dev/v1beta1
kind: ECSApp
spec:
  input:
    serviceDefinitionFile: /path/to/servicedef.yaml
    taskDefinitionFile: /path/to/taskdef.yaml
    targetGroups:
      primary:
        targetGroupArn: arn:aws:elasticloadbalancing:xyz
        containerName: web
        containerPort: 80
    listenerRuleSelector:  # HERE.   If not specified, the default rule will be changed.
      listenerRuleArns:  # Which rules to modify.
        - arn:aws:elasticloadbalancing:ap-northeast-1:<account-id>:listener-rule/app/<elb-name>/xxx/yyy/zzz
        - arn:aws:elasticloadbalancing:ap-northeast-1:<account-id>:listener-rule/app/<elb-name>/xxx/yyy/zzz2

[moko-poi]

@t-kikuc
I've considered a similar implementation in the past. On August 31, 2023, during the PipeCD Public Meeting, we received feedback suggesting that instead of receiving configuration from app.pipecd.yaml, it would be preferable to integrate these changes into the existing ModifyListener structure. You can find the meeting notes and discussion here:

Meeting Notes
YouTube Video

Additionally, in a previous implementation attempt, we had set it up to receive configurations from piped, as you can see in this comment: #4540 (comment).

[t-kikuc]

@moko-poi
Thank you so much, I see.

You mean when app.pipecd.yaml is as below without listenerRuleArns,
Default and rule-hoge should be modified because their target groups are the same as specified ones, right?

Then our goal is to modify not only the default action but also the rules which have the same target groups without specifying in app.pipecd.yaml, right?

apiVersion: pipecd.dev/v1beta1
kind: ECSApp
spec:
  input:
    serviceDefinitionFile: /path/to/servicedef.yaml
    taskDefinitionFile: /path/to/taskdef.yaml
    targetGroups:
      primary:
        targetGroupArn: arn:aws:elasticloadbalancing:xyz  # arn of `t-kikuc-ecs-simple-target-group`
        containerName: web
        containerPort: 80
      canary:
        targetGroupArn: arn:aws:elasticloadbalancing:xyz  # arn of `t-kikuc-ecs-simple-target-group2`
        containerName: web
        containerPort: 80

[moko-poi]

Absolutely, that's correct.
Our aim is to update not just the default action but also any associated rules for the same target groups, even if they aren't specified in app.pipecd.yaml.

[t-kikuc]

@moko-poi
Thanks so much.
I agree with you, we don't want to specify so much in app.pipecd.yaml.
This is my PR for that : #4733

Off course, if there are users who have listener rules with the same targets but don't want change some of them,
we should add configs like #4726 and enable to except them manually.

[t-kikuc]

@moko-poi
We merged #4733 (config-less) and closed #4726 (with config).
So may I close this discussion?
If you have any ideas after releasing it as rc-version, please feel free to make another issue/discussion!

[t-kikuc]

We released v0.46.0-rc2, which includes #4733 (config-less)