bloglike:2021-06
Differences
This shows you the differences between two versions of the page.
Next revision | Previous revision | ||
bloglike:2021-06 [2021/06/04 09:18] – AWS CodeBuild and CloudWatch Logs integration stybla | bloglike:2021-06 [2021/06/29 03:51] (current) – How to run django-q or similar at AWS Elastic Beanstalk stybla | ||
---|---|---|---|
Line 9: | Line 9: | ||
It looks alright and usable, especially logs available in build status. Better than fishing logs from S3. | It looks alright and usable, especially logs available in build status. Better than fishing logs from S3. | ||
+ | |||
+ | --- // | ||
+ | |||
+ | |||
+ | ===== When AWS CodeBuild and CloudWatch Logs integration doesn' | ||
+ | |||
+ | Previously, I was praising AWS CodeBuild and CloudWatch Logs integration. Today, it's something else :) I was in need of quick-and-dirty CodeBuild project in order to try something. I've made couple mistakes along the way, but I still think and it's possible there are cases when logs won't appear in CloudWatch and when you still need to offload logs to S3 bucket. | ||
+ | |||
+ | < | ||
+ | [Container] 2021/06/14 15:05:28 Waiting for agent ping | ||
+ | [Container] 2021/06/14 15:05:31 Waiting for DOWNLOAD_SOURCE | ||
+ | [Container] 2021/06/14 15:05:32 Phase is DOWNLOAD_SOURCE | ||
+ | [Container] 2021/06/14 15:05:32 CODEBUILD_SRC_DIR=/ | ||
+ | [Container] 2021/06/14 15:05:32 YAML location is / | ||
+ | [Container] 2021/06/14 15:05:32 Phase complete: DOWNLOAD_SOURCE State: FAILED | ||
+ | [Container] 2021/06/14 15:05:32 Phase context status code: YAML_FILE_ERROR Message: wrong number of container tags, expected 1 | ||
+ | </ | ||
+ | |||
+ | |||
+ | btw there was no '' | ||
+ | |||
+ | < | ||
+ | [Container] 2021/06/14 15:46:59 Waiting for agent ping | ||
+ | [Container] 2021/06/14 15:47:02 Waiting for DOWNLOAD_SOURCE | ||
+ | mounting ' | ||
+ | </ | ||
+ | |||
+ | I had to setup S3 bucket(so much for quick) in order to get to the bottom of what exactly is wrong. | ||
+ | |||
+ | --- // | ||
+ | |||
+ | |||
+ | ===== It's a trap! and its name is FIFO queue ===== | ||
+ | |||
+ | When migrating from Redis-as-a-queue to AWS SQS and making application more integrated with the cloud, we've decided to use a FIFO queue. Intentions were good - ideally keep order of messages and prevent duplicates as low as possible. We were aware of the following excerpt from FAQ: | ||
+ | |||
+ | > By design, Amazon SQS FIFO queues don't serve messages from the same message group to more than one consumer at a time. However, if your FIFO queue has multiple message groups, you can take advantage of parallel consumers | ||
+ | |||
+ | but not so much of implications, | ||
+ | |||
+ | As for the implications(which I haven' | ||
+ | |||
+ | Therefore, we've switched to the simple SQS queue with no content deduplication(which probably didn't work anyway with this setup) and no FIFO and possibility of duplicates. However, that's life and it works. | ||
+ | |||
+ | --- // | ||
+ | |||
+ | |||
+ | ===== How to run django-q or similar at AWS Elastic Beanstalk ===== | ||
+ | |||
+ | This is like one year late and doesn' | ||
+ | |||
+ | EBT offers "web application" | ||
+ | |||
+ | > When you launch a worker environment, | ||
+ | |||
+ | Daemon passing messages from the queue as HTTP requests? That won't fly with django-q(at least I'm not aware such thing is possible) and it doesn' | ||
+ | |||
+ | Ok, " | ||
+ | |||
+ | --- // |
bloglike/2021-06.1622816315.txt.gz · Last modified: 2021/06/04 09:18 by stybla