Sadly, this isn't something I've discovered myself. However, it's hard to figure out and info hard to come by. You setup AWS EventBridge(formerly CloudWatch Events) rule with CloudWatch log group as a target by any means except web console. Events start flowing in and then - nothing. All you get are failed invocations, resp. FailedInvocation metric going up. EventBridge seems to be hard to debug since there are no logs what so ever which is supported here. Quote:

I just confirmed from internal cloudwatch team, cloudwatch do not provide any logs for failed invocation. Apart from the failedinvocation metrics, there is no logging avaialble from cloudwatch side. As mentioned, you need to rely on lambda logs or resources logs.

Failed invocation can be caused by event pattern, input transformer, input template, IAM policy in case of Lambda and ECS - you name it. Or it can be caused by a missing CloudWatch log resource policy.

data "aws_iam_policy_document" "trust_events_to_store_log_event" {
  statement {
    sid = "TrustEventsToStoreLogEvent"

    actions = [
      "logs:CreateLogStream",
      "logs:PutLogEvents"
    ]

    resources = [
      "arn:aws:logs:us-east-1:111111111111:log-group:/aws/events/*:*"
    ]

    principals {
      identifiers = [
        "delivery.logs.amazonaws.com",
        "events.amazonaws.com"
      ]
      type = "Service"
    }
  }

  version = "2012-10-17"
}

resource "aws_cloudwatch_log_resource_policy" "trust_events_to_store_log_event" {
  policy_document = data.aws_iam_policy_document.trust_events_to_store_log_event.json
  policy_name     = "TrustEventsToStoreLogEvent"
}

As it turns out such policy is added(behind the scenes, of course) when you setup things through web console. And as far as I know these policies aren't visible anywhere in web console(ok, I haven't figured out where). One can list these policies through AWS CLI by aws logs describe-resource-policies, though. Once this policy is set, even by conducting some tests in the early life of an AWS account, through web console things magically work. However, it can be very confusing when things don't work and all you know it must be some kind of magic, but you don't know what kind.

See this GitHub issue for more info.

— Zdenek Styblik 2022/05/14 09:08

There is not much to write about except it works, at least in 2022. It seems that wasn't always the case. You will need to connect accounts by either VPC peering or Transit gateway or however you like and need. Then there is a handy how-to/KB article from AWS which seems a bit overcomplicated than it needs to be. Despite I like CLI, you can find IP addresses in the web console which seems way easier and faster than querying an internal API to which you must have ACLs. You need this in order to modify /etc/hosts.

And that should be it. Of course, everything depends on your setup, security etc. One last thing. Should you encounter Connection reset by peer, it most likely means that VPC range from the other account needs to be allowed in a security group attached to EFS mount target/access point.

— Zdenek Styblik 2022/05/16 08:01