Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

RDS Cluster Parameter Group violates plan even without changes #4997

Open
kaique-paypal opened this issue Dec 23, 2024 · 3 comments
Open

RDS Cluster Parameter Group violates plan even without changes #4997

kaique-paypal opened this issue Dec 23, 2024 · 3 comments
Labels
awaiting-feedback Blocked on input from the author kind/bug Some behavior is incorrect or out of spec needs-repro Needs repro steps before it can be triaged or fixed

Comments

@kaique-paypal
Copy link

Describe what happened

We recently enabled Pulumi "plan" in our CI/CD and we're facing an issue where an RDS cluster parameter group is violating the plan even when the changes are not related to this resource.
We are setting this parameters to our parameter group:

    parameters:
      param-1:
        name: "pg_stat_statements.track"
        value: "ALL"
      param-2:
        name: "pg_stat_statements.max"
        value: "10000"
        applyMethod: "pending-reboot"
      param-3:
        name: "shared_preload_libraries"
        value: "pgaudit,pg_stat_statements"
        applyMethod: "pending-reboot"
      param-4:
        name: "rds.logical_replication"
        value: "1"
        applyMethod: "pending-reboot"
      param-5:
        name: "wal_sender_timeout"
        value: "0"
      param-6:
        name: "max_replication_slots"
        value: "30"
        applyMethod: "pending-reboot"
      param-7:
        name: "idle_in_transaction_session_timeout"
        value: "1800000"
        applyMethod: "immediate"
      param-8:
        name: "max_wal_senders"
        value: "30"
        applyMethod: "pending-reboot"
      param-9:
        name: "pgaudit.role"
        value: "rds_pgaudit"
      param-10:
        name: "pgaudit.log"
        value: "all"
      param-11:
        name: "rds.force_ssl"
        value: "1"

The output error after running pulumi up --plan='plan.json'
error: resource urn:pulumi:infra-happyreturns-legacy-dev::aws-platform::hr:aws:ClusterParameterGroup$aws:rds/clusterParameterGroup:ClusterParameterGroup::dev-happyreturns-13/ClusterParameterGroup violates plan: properties changed: ~~parameters[{[{map[__defaults:{[{applyMethod}]} applyMethod:{immediate} name:{pgaudit.log} value:{all}]} {map[__defaults:{[]} applyMethod:{pending-reboot} name:{max_wal_senders} value:{30}]} {map[__defaults:{[]} applyMethod:{pending-reboot} name:{max_replication_slots} value:{30}]} {map[__defaults:{[]} applyMethod:{immediate} name:{idle_in_transaction_session_timeout} value:{1800000}]} {map[__defaults:{[{applyMethod}]} applyMethod:{immediate} name:{pgaudit.role} value:{rds_pgaudit}]} {map[__defaults:{[{applyMethod}]} applyMethod:{immediate} name:{wal_sender_timeout} value:{0}]} {map[__defaults:{[{applyMethod}]} applyMethod:{immediate} name:{rds.force_ssl} value:{1}]} {map[__defaults:{[]} applyMethod:{pending-reboot} name:{rds.logical_replication} value:{1}]} {map[__defaults:{[]} applyMethod:{pending-reboot} name:{pg_stat_statements.max} value:{10000}]} {map[__defaults:{[]} applyMethod:{pending-reboot} name:{shared_preload_libraries} value:{pgaudit,pg_stat_statements}]} {map[__defaults:{[{applyMethod}]} applyMethod:{immediate} name:{pg_stat_statements.track} value:{ALL}]}]}!={[{map[__defaults:{[]} applyMethod:{pending-reboot} name:{max_replication_slots} value:{30}]} {map[__defaults:{[]} applyMethod:{immediate} name:{idle_in_transaction_session_timeout} value:{1800000}]} {map[__defaults:{[]} applyMethod:{pending-reboot} name:{rds.logical_replication} value:{1}]} {map[__defaults:{[{applyMethod}]} applyMethod:{immediate} name:{wal_sender_timeout} value:{0}]} {map[__defaults:{[{applyMethod}]} applyMethod:{immediate} name:{pgaudit.log} value:{all}]} {map[__defaults:{[]} applyMethod:{pending-reboot} name:{pg_stat_statements.max} value:{10000}]} {map[__defaults:{[]} applyMethod:{pending-reboot} name:{max_wal_senders} value:{30}]} {map[__defaults:{[{applyMethod}]} applyMethod:{immediate} name:{rds.force_ssl} value:{1}]} {map[__defaults:{[{applyMethod}]} applyMethod:{immediate} name:{pgaudit.role} value:{rds_pgaudit}]} {map[__defaults:{[{applyMethod}]} applyMethod:{immediate} name:{pg_stat_statements.track} value:{ALL}]} {map[__defaults:{[]} applyMethod:{pending-reboot} name:{shared_preload_libraries} value:{pgaudit,pg_stat_statements}]}]}]

The pulumi plan file:

        "urn:pulumi:infra-happyreturns-legacy-dev::aws-platform::hr:aws:ClusterParameterGroup$aws:rds/clusterParameterGroup:ClusterParameterGroup::dev-happyreturns-13/ClusterParameterGroup": {
            "goal": {
                "type": "aws:rds/clusterParameterGroup:ClusterParameterGroup",
                "name": "dev-happyreturns-13/ClusterParameterGroup",
                "custom": true,
                "inputDiff": {
                    "updates": {
                        "parameters": [
                            {
                                "__defaults": [
                                    "applyMethod"
                                ],
                                "applyMethod": "immediate",
                                "name": "pgaudit.log",
                                "value": "all"
                            },
                            {
                                "__defaults": [],
                                "applyMethod": "pending-reboot",
                                "name": "max_wal_senders",
                                "value": "30"
                            },
                            {
                                "__defaults": [],
                                "applyMethod": "pending-reboot",
                                "name": "max_replication_slots",
                                "value": "30"
                            },
                            {
                                "__defaults": [],
                                "applyMethod": "immediate",
                                "name": "idle_in_transaction_session_timeout",
                                "value": "1800000"
                            },
                            {
                                "__defaults": [
                                    "applyMethod"
                                ],
                                "applyMethod": "immediate",
                                "name": "pgaudit.role",
                                "value": "rds_pgaudit"
                            },
                            {
                                "__defaults": [
                                    "applyMethod"
                                ],
                                "applyMethod": "immediate",
                                "name": "wal_sender_timeout",
                                "value": "0"
                            },
                            {
                                "__defaults": [
                                    "applyMethod"
                                ],
                                "applyMethod": "immediate",
                                "name": "rds.force_ssl",
                                "value": "1"
                            },
                            {
                                "__defaults": [],
                                "applyMethod": "pending-reboot",
                                "name": "rds.logical_replication",
                                "value": "1"
                            },
                            {
                                "__defaults": [],
                                "applyMethod": "pending-reboot",
                                "name": "pg_stat_statements.max",
                                "value": "10000"
                            },
                            {
                                "__defaults": [],
                                "applyMethod": "pending-reboot",
                                "name": "shared_preload_libraries",
                                "value": "pgaudit,pg_stat_statements"
                            },
                            {
                                "__defaults": [
                                    "applyMethod"
                                ],
                                "applyMethod": "immediate",
                                "name": "pg_stat_statements.track",
                                "value": "ALL"
                            }
                        ]
                    }
                },
                "outputDiff": {},
                "parent": "urn:pulumi:infra-happyreturns-legacy-dev::aws-platform::hr:aws:ClusterParameterGroup::dev-happyreturns-13",
                "protect": false,
                "provider": "urn:pulumi:infra-happyreturns-legacy-dev::aws-platform::pulumi:providers:aws::default_6_65_0::e8f0ea2d-b2a4-4641-8caf-67a5e4ffd5f6",
                "propertyDependencies": {
                    "description": [],
                    "family": [],
                    "name": [],
                    "parameters": [],
                    "tags": []
                },
                "deleteBeforeReplace": true,
                "customTimeouts": {}
            },
            "steps": [
                "same"
            ],
            "state": {
                "arn": "arn:aws:rds:us-west-2:374812476261:cluster-pg:dev-happyreturns-13",
                "description": "Managed by Pulumi",
                "family": "aurora-postgresql13",
                "id": "dev-happyreturns-13",
                "name": "dev-happyreturns-13",
                "namePrefix": "",
                "parameters": [
                    {
                        "applyMethod": "immediate",
                        "name": "idle_in_transaction_session_timeout",
                        "value": "1800000"
                    },
                    {
                        "applyMethod": "immediate",
                        "name": "pg_stat_statements.track",
                        "value": "ALL"
                    },
                    {
                        "applyMethod": "immediate",
                        "name": "pgaudit.log",
                        "value": "all"
                    },
                    {
                        "applyMethod": "immediate",
                        "name": "pgaudit.role",
                        "value": "rds_pgaudit"
                    },
                    {
                        "applyMethod": "immediate",
                        "name": "rds.force_ssl",
                        "value": "1"
                    },
                    {
                        "applyMethod": "immediate",
                        "name": "wal_sender_timeout",
                        "value": "0"
                    },
                    {
                        "applyMethod": "pending-reboot",
                        "name": "max_replication_slots",
                        "value": "30"
                    },
                    {
                        "applyMethod": "pending-reboot",
                        "name": "max_wal_senders",
                        "value": "30"
                    },
                    {
                        "applyMethod": "pending-reboot",
                        "name": "pg_stat_statements.max",
                        "value": "10000"
                    },
                    {
                        "applyMethod": "pending-reboot",
                        "name": "rds.logical_replication",
                        "value": "1"
                    },
                    {
                        "applyMethod": "pending-reboot",
                        "name": "shared_preload_libraries",
                        "value": "pgaudit,pg_stat_statements"
                    }
                ],
                "tags": {
                    "contact": "[email protected]",
                    "env": "dev",
                    "hr:compliance:Framework": "SOC2",
                    "hr:cost:BusinessUnitID": "Engineering",
                    "hr:cost:CostCenter": "happyreturns",
                    "hr:cost:Owner": "CloudInfrastructure",
                    "hr:ops:Contact": "[email protected]",
                    "hr:ops:Environment": "dev",
                    "hr:ops:IAC": "Pulumi",
                    "hr:ops:Owner": "CloudInfrastructure",
                    "hr:ops:hr:ops:ManagedBy": "aws-platform/infra-happyreturns-legacy-dev",
                    "pulumi-stack": "aws-platform/infra-happyreturns-legacy-dev",
                    "team": "devops"
                },
                "tagsAll": {
                    "contact": "[email protected]",
                    "env": "dev",
                    "hr:compliance:Framework": "SOC2",
                    "hr:cost:BusinessUnitID": "Engineering",
                    "hr:cost:CostCenter": "happyreturns",
                    "hr:cost:Owner": "CloudInfrastructure",
                    "hr:ops:Contact": "[email protected]",
                    "hr:ops:Environment": "dev",
                    "hr:ops:IAC": "Pulumi",
                    "hr:ops:Owner": "CloudInfrastructure",
                    "hr:ops:hr:ops:ManagedBy": "aws-platform/infra-happyreturns-legacy-dev",
                    "pulumi-stack": "aws-platform/infra-happyreturns-legacy-dev",
                    "team": "devops"
                }
            },
            "seed": "a++32PxL/DJpdZBkM4CHFPgCG6z6VA0tg3t+gbWC5A0="
        },
```

### Sample program

```
package main

import (
	"github.com/pulumi/pulumi-aws/sdk/v6/go/aws/rds"
	"github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
	pulumi.Run(func(ctx *pulumi.Context) error {
		_, err := rds.NewClusterParameterGroup(ctx, "default", &rds.ClusterParameterGroupArgs{
			Name:        pulumi.String("rds-cluster-pg"),
			Family:      pulumi.String("aurora-postgresql13"),
			Description: pulumi.String("RDS default cluster parameter group"),
			Parameters: rds.ClusterParameterGroupParameterArray{
				&rds.ClusterParameterGroupParameterArgs{
					Name:  pulumi.String("pg_stat_statements.track"),
					Value: pulumi.String("ALL"),
				},
				&rds.ClusterParameterGroupParameterArgs{
					Name:  pulumi.String("pg_stat_statements.max"),
					Value: pulumi.String("10000"),
					ApplyMethod: pulumi.String("pending-reboot"),
				},
				&rds.ClusterParameterGroupParameterArgs{
					Name:  pulumi.String("shared_preload_libraries"),
					Value: pulumi.String("pgaudit,pg_stat_statements"),
					ApplyMethod: pulumi.String("pending-reboot"),
				},
				&rds.ClusterParameterGroupParameterArgs{
					Name:  pulumi.String("rds.logical_replication"),
					Value: pulumi.String("1"),
					ApplyMethod: pulumi.String("pending-reboot"),
				},
				&rds.ClusterParameterGroupParameterArgs{
					Name:  pulumi.String("wal_sender_timeout"),
					Value: pulumi.String("0"),
				},
				&rds.ClusterParameterGroupParameterArgs{
					Name:  pulumi.String("max_replication_slots"),
					Value: pulumi.String("30"),
					ApplyMethod: pulumi.String("pending-reboot"),
				},
				&rds.ClusterParameterGroupParameterArgs{
					Name:  pulumi.String("idle_in_transaction_session_timeout"),
					Value: pulumi.String("1800000"),
					ApplyMethod: pulumi.String("immediate"),
				},
				&rds.ClusterParameterGroupParameterArgs{
					Name:  pulumi.String("max_wal_senders"),
					Value: pulumi.String("30"),
					ApplyMethod: pulumi.String("pending-reboot"),
				},
				&rds.ClusterParameterGroupParameterArgs{
					Name:  pulumi.String("pgaudit.role"),
					Value: pulumi.String("rds_pgaudit"),
				},
				&rds.ClusterParameterGroupParameterArgs{
					Name:  pulumi.String("pgaudit.log"),
					Value: pulumi.String("all"),
				},
				&rds.ClusterParameterGroupParameterArgs{
					Name:  pulumi.String("rds.force_ss"),
					Value: pulumi.String("1"),
				},
			},
		})
		if err != nil {
			return err
		}
		return nil
	})
}
```

### Log output

_No response_

### Affected Resource(s)

_No response_

### Output of `pulumi about`

```
CLI
Version      3.143.0
Go Version   go1.23.4
Go Compiler  gc

Plugins
KIND      NAME        VERSION
resource  aws         6.65.0
resource  aws-native  1.4.0
language  go          3.143.0
resource  kubernetes  4.14.0
resource  random      4.16.3
resource  tls         4.11.1

Host
OS       darwin
Version  14.7
Arch     arm64

This project is written in go: executable='/opt/homebrew/bin/go' version='go version go1.23.2 darwin/arm64'

warning: Could not access the backend: read ".pulumi/meta.yaml": blob (key ".pulumi/meta.yaml") (code=Unknown): operation error S3: GetObject, https response error StatusCode: 400, RequestID: 4S0ZJW1YHHT7GEH4, HostID: OvhDtfNj/i3M7lYXadC0mDzirK7G2GG5SRopkwS6yWHxPOrNTrHi9QNgzEA2hKi3ZagVltpGnoPagkkXHZfa6g==, api error ExpiredToken: The provided token has expired.
Dependencies:
NAME                                        VERSION
github.com/go-viper/mapstructure/v2         v2.0.0
github.com/knadh/koanf                      v1.5.0
github.com/knadh/koanf/v2                   v2.1.1
github.com/pulumi/pulumi-aws-native/sdk     v1.4.0
github.com/pulumi/pulumi-aws/sdk/v6         v6.65.0
github.com/pulumi/pulumi-kubernetes/sdk/v4  v4.14.0
github.com/pulumi/pulumi/sdk/v3             v3.142.0
github.com/stretchr/testify                 v1.9.0
golang.org/x/exp                            v0.0.0-20240707233637-46b078467d37
github.com/pulumi/pulumi-random/sdk/v4      v4.16.3
github.com/pulumi/pulumi-tls/sdk/v4         v4.11.1
github.com/sethvargo/go-githubactions       v1.3.0
github.com/valyala/fastjson                 v1.6.4
```

### Additional context

_No response_

### Contributing

Vote on this issue by adding a 👍 reaction. 
To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).
@kaique-paypal kaique-paypal added kind/bug Some behavior is incorrect or out of spec needs-triage Needs attention from the triage team labels Dec 23, 2024
@flostadler
Copy link
Contributor

Hey @kaique-paypal, sorry that you're running into issues here!
I tried reproducing it with your example (minor change - "rds.force_ssl" vs "rds.force_ss") and it worked as expected for me. Are there any specific steps I should follow to trigger this behavior?

I was running:

  • pulumi preview --save-plan=plan.json
  • pulumi up --plan='plan.json'

These are the versions I was using:

CLI
Version      3.136.1
Go Version   go1.23.2
Go Compiler  gc

Plugins
KIND      NAME  VERSION
resource  aws   6.66.1
language  go    3.136.1

Host
OS       darwin
Version  15.2
Arch     arm64

This project is written in go: executable='/opt/homebrew/bin/go' version='go version go1.23.3 darwin/arm64'

...

Dependencies:
NAME                                 VERSION
github.com/pulumi/pulumi-aws/sdk/v6  v6.66.1
github.com/pulumi/pulumi/sdk/v3      v3.144.1

@flostadler flostadler added needs-repro Needs repro steps before it can be triaged or fixed awaiting-feedback Blocked on input from the author and removed needs-triage Needs attention from the triage team labels Dec 24, 2024
@kaique-paypal
Copy link
Author

oh sorry about the typo 🤦
Probably one thing to try is to create the resource beforehand and then try to run any other update using plan. This is exactly what we did.
The resources already existed when we implemented the plan usage.

@pulumi-bot pulumi-bot added needs-triage Needs attention from the triage team and removed awaiting-feedback Blocked on input from the author labels Dec 24, 2024
@t0yv0
Copy link
Member

t0yv0 commented Dec 30, 2024

Could you please provide a detailed repro?

Using this code:

package main

import (
	"github.com/pulumi/pulumi-aws/sdk/v6/go/aws/rds"
	"github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
	pulumi.Run(func(ctx *pulumi.Context) error {
		_, err := rds.NewClusterParameterGroup(ctx, "default", &rds.ClusterParameterGroupArgs{
			Name:        pulumi.String("rds-cluster-pg"),
			Family:      pulumi.String("aurora-postgresql13"),
			Description: pulumi.String("RDS default cluster parameter group"),
			Parameters: rds.ClusterParameterGroupParameterArray{
				&rds.ClusterParameterGroupParameterArgs{
					Name:  pulumi.String("pg_stat_statements.track"),
					Value: pulumi.String("ALL"),
				},
				&rds.ClusterParameterGroupParameterArgs{
					Name:        pulumi.String("pg_stat_statements.max"),
					Value:       pulumi.String("10000"),
					ApplyMethod: pulumi.String("pending-reboot"),
				},
				&rds.ClusterParameterGroupParameterArgs{
					Name:        pulumi.String("shared_preload_libraries"),
					Value:       pulumi.String("pgaudit,pg_stat_statements"),
					ApplyMethod: pulumi.String("pending-reboot"),
				},
				&rds.ClusterParameterGroupParameterArgs{
					Name:        pulumi.String("rds.logical_replication"),
					Value:       pulumi.String("1"),
					ApplyMethod: pulumi.String("pending-reboot"),
				},
				&rds.ClusterParameterGroupParameterArgs{
					Name:  pulumi.String("wal_sender_timeout"),
					Value: pulumi.String("0"),
				},
				&rds.ClusterParameterGroupParameterArgs{
					Name:        pulumi.String("max_replication_slots"),
					Value:       pulumi.String("30"),
					ApplyMethod: pulumi.String("pending-reboot"),
				},
				&rds.ClusterParameterGroupParameterArgs{
					Name:        pulumi.String("idle_in_transaction_session_timeout"),
					Value:       pulumi.String("1800000"),
					ApplyMethod: pulumi.String("immediate"),
				},
				&rds.ClusterParameterGroupParameterArgs{
					Name:        pulumi.String("max_wal_senders"),
					Value:       pulumi.String("30"),
					ApplyMethod: pulumi.String("pending-reboot"),
				},
				&rds.ClusterParameterGroupParameterArgs{
					Name:  pulumi.String("pgaudit.role"),
					Value: pulumi.String("rds_pgaudit"),
				},
				&rds.ClusterParameterGroupParameterArgs{
					Name:  pulumi.String("pgaudit.log"),
					Value: pulumi.String("all"),
				},
				&rds.ClusterParameterGroupParameterArgs{
					Name:  pulumi.String("rds.force_ssl"),
					Value: pulumi.String("1"),
				},
			},
		})
		if err != nil {
			return err
		}
		return nil
	})
}

And the following steps:

pulumi up --yes

Apply an edit:

				&rds.ClusterParameterGroupParameterArgs{
					Name:        pulumi.String("max_wal_senders"),
					Value:       pulumi.String("20"),
					ApplyMethod: pulumi.String("pending-reboot"),
				},

Then:

pulumi preview --save-plan=plan.json
pulumi up --plan='plan.json'

Applies without issue for me.

@t0yv0 t0yv0 added awaiting-feedback Blocked on input from the author and removed needs-triage Needs attention from the triage team labels Dec 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
awaiting-feedback Blocked on input from the author kind/bug Some behavior is incorrect or out of spec needs-repro Needs repro steps before it can be triaged or fixed
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@t0yv0 @flostadler @pulumi-bot @kaique-paypal