aws:click

About

Adapted from the Aws tutorial - Building a Modern app (module 5).

Articles Related

Architecture

Click Event records will be processed in real-time by a serverless code function, aggregated, and stored for any future analysis that you may want to perform.

Steps

Create the code and save it to a repository

aws codecommit create-repository --repository-name MythicalMysfitsStreamingService-Repository

{
    "repositoryMetadata": {
        "accountId": "REPLACE_ME_ACCOUNT_ID",
        "repositoryId": "32941d10-66e1-4230-b3d6-eeb55e64a60b",
        "repositoryName": "MythicalMysfitsStreamingService-Repository",
        "lastModifiedDate": 1554965738.429,
        "creationDate": 1554965738.429,
        "cloneUrlHttp": "https://git-codecommit.eu-central-1.amazonaws.com/v1/repos/MythicalMysfitsStreamingService-Repository",
        "cloneUrlSsh": "ssh://git-codecommit.eu-central-1.amazonaws.com/v1/repos/MythicalMysfitsStreamingService-Repository",
        "Arn": "arn:aws:codecommit:eu-central-1:REPLACE_ME_ACCOUNT_ID:MythicalMysfitsStreamingService-Repository"
    }
}

codecommit credential setup
Clone

git clone https://git-codecommit.eu-central-1.amazonaws.com/v1/repos/MythicalMysfitsStreamingService-Repository

Cloning into 'MythicalMysfitsStreamingService-Repository'...
warning: You appear to have cloned an empty repository.

Copy the source

cd MythicalMysfitsStreamingService-Repository
cp ..\aws-modern-application-workshop\module-5\cfn .
cp ..\aws-modern-application-workshop\module-5\app\streaming .

:: A CFN template for creating the full stack.
..\aws-modern-application-workshop\module-5\cfn\real-time-streaming.yml
        1 file(s) copied.
:: A Python script that contains the code for our Lambda function: streamProcessor.py
..\aws-modern-application-workshop\module-5\app\streaming\streamProcessor.py
        1 file(s) copied.

package all of library dependencies together with the Lambda code function

:: Install the request package into the local directory
pip install requests -t .

Collecting requests
  Using cached https://files.pythonhosted.org/packages/7d/e3/20f3d364d6c8e5d2353c72a67778eb189176f08e873c9900e10c0287b84b/requests-2.21.0-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2 (from requests)
  Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
Collecting certifi>=2017.4.17 (from requests)
  Downloading https://files.pythonhosted.org/packages/60/75/f692a584e85b7eaba0e03827b3d51f45f571c2e793dd731e598828d380aa/certifi-2019.3.9-py2.py3-none-any.whl (158kB)
    100% |                                        | 163kB 163kB/s
Collecting urllib3<1.25,>=1.21.1 (from requests)
  Using cached https://files.pythonhosted.org/packages/62/00/ee1d7de624db8ba7090d1226aebefab96a2c71cd5cfa7629d6ad3f61b79e/urllib3-1.24.1-py2.py3-none-any.whl
Collecting idna<2.9,>=2.5 (from requests)
  Using cached https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl
Installing collected packages: chardet, certifi, urllib3, idna, requests
Successfully installed certifi-2019.3.9 chardet-3.0.4 idna-2.8 requests-2.21.0 urllib3-1.24.1

List

ls -A1

.git
bin
certifi
certifi-2019.3.9.dist-info
chardet
chardet-3.0.4.dist-info
idna
idna-2.8.dist-info
real-time-streaming.yml
requests
requests-2.21.0.dist-info
streamProcessor.py
urllib3
urllib3-1.24.1.dist-info

Replace REPLACE_ME_API_ENDPOINT in the Lambda Function Code streamProcessor.py by the ApiEndpoint of the Mysfits service API (this is the same service ApiEndpoint that you created for the website frontend).

def retrieveMysfit(mysfitId):
    apiEndpoint = 'REPLACE_ME_API_ENDPOINT' + '/mysfits/' + str(
        mysfitId)  # eg: 'https://ljqomqjzbf.execute-api.us-east-1.amazonaws.com/prod/'
    mysfit = requests.get(apiEndpoint).json()
    return mysfit

Package

git add .
git commit -m "New stream processing service."
git push

Create the streaming service stack with SAM

The creation of the streaming stack is done using the AWS SAM CLI to:

package all of our function code,
upload it to S3,
and create the deployable CloudFormation template

Create a S3 bucket

Create a bucket named real-time-streaming

aws s3 mb s3://real-time-streaming

Package the code with SAM

With the Sam cli

cd aws-modern-application-workshop\module-5
sam package ^
  --template-file .\cfn\real-time-streaming.yml ^
  --output-template-file .\cfn\transformed-streaming.yml ^
  --s3-bucket real-time-streaming

Uploading to 31ae6f83bb35657cc2aad3d168246ada  2629 / 2629.0  (100.00%)
Successfully packaged artifacts and wrote output template to file .\cfn\transformed-streaming.yml.
Execute the following command to deploy the packaged template
aws cloudformation deploy --template-file d:\code-external\aws-modern-application-workshop\module-5\cfn\transformed-streaming.yml --stack-name <YOUR STACK NAME>

The CodeUri parameter of the serverless Lambda function has been updated in the transformed-streaming.yml file with the object location where the SAM CLI has uploaded your packaged code

.........
MysfitsClicksProcessor:
    Type: AWS::Serverless::Function
    Properties:
      Handler: streamProcessor.processRecord
      Runtime: python3.6
      CodeUri: s3://real-time-streaming/31ae6f83bb35657cc2aad3d168246ada
      ..........

Deploy The Stack Using AWS CloudFormation

Returned by the SAM CLI command is the CloudFormation command needed to be executed to create our new full stack. But because our stack creates IAM resources, you'll need to add one additional parameter to the command.

deploy the streaming stack (deploy the function)

aws cloudformation deploy ^
  --template-file .\cfn\transformed-streaming.yml ^
  --stack-name MythicalMysfitsStreamingStack ^
  --capabilities CAPABILITY_IAM

Once this stack creation is complete, the full real-time processing microservice will be created.

Documentation / Reference

https://aws.amazon.com/getting-started/projects/build-modern-app-fargate-lambda-dynamodb-python/module-five/

Support

AccessDenied when calling the CreateChangeSet operation

An error occurred (AccessDenied) when calling the CreateChangeSet operation: User: arn:aws:iam::09000999845148:user/cli is not authorized to perform: cloudformation:CreateChangeSet on resource: arn:aws:cloudformation:eu-central-1:094473333148:stack/MythicalMysfitsStreamingStack/*

Table of Contents

Aws - Click Event Rest API (Capture user behavior)