Dynamodb CRUD with Typescript Lambda CDK

Before we get started I would recommend to develop CDK project inside development docker container.

So we no longer have the issue of "this works on my computer". And it will not mess with your existing workflow. So checkout my blog post for detailed video explanation.

Then checkout CDK development container settings on github.

View Dynamodb Crud Source code on github

Let's create new CDK project

mkdir dynamodb-crud
cd $_
cdk init app --language typescript
npm i @aws-cdk/aws-dynamodb

So we created new folder, initialized new cdk project and installed dynamodb package.

The $_ is variable for last word of last command. In our case $_ is substituting for dynamodb-crud.

Then we are going to create a table as follows.

So we are creating new dynamodb table. The only required property for creating new table is the partition key.

And we are keeping it simple with id with type string as partition key.

Billing mode default is pay per request but I like to manually set it because the defaults may change over time and I may end up paying the price for it.

The removal policy default is retain. That means after you delete this stack the table will be orphaned.

But I would like to delete the table with the stack.

The table name is generated automatically. You can specify it manually but its better to generate automatically to maintain a unique name.

I am printing out the table name in stack output. Because we need it while performing the crud operations in lambda.

Then deploy this stack. With cdk diff and cdk deploy DynamodbCrudStack-AnBmdHH53PUgeF7R

Save the printed table name in variables.ts file as follows

export const todoTableName = 'DynamodbCrudStack-AnBmdHH53PUgeF7R-todoTable'

Then create 4 different lambda folders as follows

mkdir lambda-fns/create/
mkdir lambda-fns/getAll/
mkdir lambda-fns/getOne/
mkdir lambda-fns/update/
mkdir lambda-fns/delete/

So we have created folders for each lambda functions.

Now go inside every function folder and initialize package.json and add following dependencies

npm init -y
npm i @aws-sdk/client-dynamodb @aws-sdk/util-dynamodb
npm i -D @types/aws-lambda @types/node typescript
touch index.ts

New AWS SDK v3 for javascript supports modular packages.

So if we only require dynamodb then we don't have to install all of aws sdk. We only install of what we need.

The util-dynamodb package helps us convert json object into appropriate dynamodb attribute value. This saves us time and makes the code more human readable and clean.

When the final lambda is compiled from typescript to javascript the development dependencies are not added to the final build. This will make our lambda run faster by reducing the size.

Then install lambda packages to the cdk project as follows

cd ../..
npm i @aws-cdk/aws-lambda @aws-cdk/aws-lambda-nodejs esbuild

Create

Now using these packages we will create lambda function in the stack as follows

So we have created nodejs function. This will automatically compile typescript into javascript and also add the dependencies but not the dev dependencies.

Important we need to give this function read and write access to the table.

And we are passing table name as environment variable.

Then add the following function inside create/index.ts

First we pull out body from event. Body can be string or undefined. If its undefined then send return fail response immediately.

Then we parse the body string and pull out id, title and done variables. You could pass new id in the payload or if you have not sent id then it can be generated automatically.

So optionally you need to install uuid as follows

npm i uuid
npm i -D @types/uuid

Then create new dynamo client and create put item input. This is where util package comes handy.

You don't have to provide attribute value type. You simply add json data and it will automatically set the type for you.

Then you save the data using client. The return value can either be ALL_OLD or NONE.

So I am create new todo object and sending that back instead of response.

Now create the event folder and file as follows.

mkdir events
touch events/create.json

Then add following event in create.json

{ "body": "{ \"title\": \"get milk\", \"done\": false }" }

Then follow my other tutorial to run lambda locally and run this function.

You will see the following output.

START RequestId: eb6b2259-23b7-4b39-9169-c591966c147a Version: $LATEST
END RequestId: eb6b2259-23b7-4b39-9169-c591966c147a
REPORT RequestId: eb6b2259-23b7-4b39-9169-c591966c147a Init Duration: 0.25 ms Duration: 1077.17 ms Billed Duration: 1100 ms Memory Size: 128 MB Max Memory Used: 128 MB
{"statusCode":200,"body":"{\"newTodo\":{\"id\":\"c67f1017-1a63-4fba-97db-3dd0f1434fc0\",\"title\":\"get milk\",\"done\":false}}"}

We got the expected response. Now next we are going to get list of todo.

So create another todo with title "get coffee" as follows

END RequestId: 38b03355-412d-4e54-8b5c-598aa5bfdaaf
REPORT RequestId: 38b03355-412d-4e54-8b5c-598aa5bfdaaf Init Duration: 0.07 ms Duration: 1056.83 ms Billed Duration: 1100 ms Memory Size: 128 MB Max Memory Used: 128 MB
{"statusCode":200,"body":"{\"newTodo\":{\"id\":\"7699e070-bbde-4ff3-b2b7-a84bf5d5297a\",\"title\":\"get coffee\",\"done\":false}}"}

Read

Then create function for getting all todo as follows

This is exactly same as before except this function has only read access to the table. This function cannot modify the table data.

Then we add actual lambda function as follows.

This time we are running scan operation on the table. This is very expensive operation.

Because it will read every single record from the table. I will add bonus at the end for query operation.

Query operation is relatively cheaper than scan because it reads fewer records to get the list. In general scan operation is NOT RECOMMENDED.

So the received array needs to be converted into json before sending it back. This is where unmarshall method from util package comes handy.

Again run lambda locally to see the following output

END RequestId: 258ad010-d313-4f42-adc1-4e53e1d2c902
REPORT RequestId: 258ad010-d313-4f42-adc1-4e53e1d2c902 Init Duration: 0.19 ms Duration: 1063.51 ms Billed Duration: 1100 ms Memory Size: 128 MB Max Memory Used: 128 MB
{"statusCode":200,"body":"{\"listTodo\":[{\"id\":\"c67f1017-1a63-4fba-97db-3dd0f1434fc0\",\"done\":false,\"title\":\"get milk\"},{\"id\":\"7699e070-bbde-4ff3-b2b7-a84bf5d5297a\",\"done\":false,\"title\":\"get coffee\"}]}"}

So in output we got both the todo items that we saved in previous steps.

Now we are going to fetch single todo using its id.

Add the following to cdk stack to create new lambda function.

This function is same as getAll function with changing name to getOne.

Then we write lambda function as follows.

This time we are parsing id from event body. And using that id we are going to fetch the todo.

This time we get to use both marshall and unmarshall methods from util package.

We need to pass in input event. So create new file touch events/getOne.json and add the following content

{ "body": "{ \"id\": \"c67f1017-1a63-4fba-97db-3dd0f1434fc0\" }" }

If we run the above function locally then we get the following output.

END RequestId: c68d2f0f-580c-4c3a-b7be-7bcc4a78c85e
REPORT RequestId: c68d2f0f-580c-4c3a-b7be-7bcc4a78c85e Init Duration: 0.12 ms Duration: 1071.82 ms Billed Duration: 1100 ms Memory Size: 128 MB Max Memory Used: 128 MB
{"statusCode":200,"body":"{\"todo\":{\"id\":\"c67f1017-1a63-4fba-97db-3dd0f1434fc0\",\"done\":false,\"title\":\"get milk\"}}"}

We got expected get milk single todo in the response.

Update

Now lets assume we went to the supermarket and bought the milk from our todo list. Now we need to update the done property to true.

So lets add new update function to the stack as follows

This time we are updating table and reading from table. So make sure to give the function read and write access to the table.

Similar to how we did for create todo function.

Then we add the actual update lambda function as follows

Here we again check for event body. If body is undefined then we send error with invalid request.

This function is specifically for updating done property. However you can also update the title of todo the same way.

Updating dynamodb is little more complex than creating or reading item.

If you want simple approach then you can also update with putItem request we did for creating new todo.

The down side of put item is that it will replace the entire object with new object.

So if you just wanted to update done property then it will delete the title of the todo.

Benefit of this complex update function is that it will only update a single property of big object without disturbing rest of the object.

For this we need to write update expression. This update expression can allow you to write complex expression.

So this way has lot of flexibility. And variables inside update expression can be written with expression attribute values.

The default return value is NONE. But we want all updated complete new object. So I have set it to ALL_NEW.

Then from the response we need to convert attribute value into simple json format. And send it back to the client.

Now create new event input file with touch events/update.json and add following content

{ "body": "{ \"id\": \"c67f1017-1a63-4fba-97db-3dd0f1434fc0\", \"done\": true }" }

If you run update function locally with update.json then you will see following output

END RequestId: f5a7ca07-d0b9-48d4-90a3-c269637f1e8e
REPORT RequestId: f5a7ca07-d0b9-48d4-90a3-c269637f1e8e Init Duration: 0.38 ms Duration: 1055.92 ms Billed Duration: 1100 ms Memory Size: 128 MB Max Memory Used: 128 MB
{"statusCode":200,"body":"{\"todo\":{\"id\":\"c67f1017-1a63-4fba-97db-3dd0f1434fc0\",\"done\":true,\"title\":\"get milk\"}}"}

Only the done value updated and entire object is received back to the client.

Delete

Now that we have finished our todo maybe we want to delete this item from our list.

So lets add a new function to our stack as follows

Just like all previous function we change name and create delete function.

Then give this function read and write access to the table.

Then add function as follows

This function is very similar to getOne function.

But we are deleting the object for given id before sending it back. So input part is the same.

This time we have to specify to return ALL_OLD values. Because by default it returns NONE values back after delete.

Similar to how we got value back from update we get the deleted object back. That we send back to the client.

We can reuse the events/getOne.json as input for delete function.

But just to be safe I am going to create new input at events/delete.json and add following content.

{ "body": "{ \"id\": \"c67f1017-1a63-4fba-97db-3dd0f1434fc0\" }" }

Now if we run delete function locally with delete.json input event then we get following output.

START RequestId: 4cd4b5b4-b9d9-4feb-bf5f-bfaa76bba1dc Version: $LATEST
END RequestId: 4cd4b5b4-b9d9-4feb-bf5f-bfaa76bba1dc
REPORT RequestId: 4cd4b5b4-b9d9-4feb-bf5f-bfaa76bba1dc Init Duration: 0.20 ms Duration: 1066.96 ms Billed Duration: 1100 ms Memory Size: 128 MB Max Memory Used: 128 MB
{"statusCode":200,"body":"{\"todo\":{\"id\":\"c67f1017-1a63-4fba-97db-3dd0f1434fc0\",\"done\":true,\"title\":\"get milk\"}}"}

Now just to verify the todo is deleted; if we run scan/getAll function then we will get following output

END RequestId: 552b989a-07f9-453e-897f-471e827c661b
REPORT RequestId: 552b989a-07f9-453e-897f-471e827c661b Init Duration: 0.09 ms Duration: 1039.45 ms Billed Duration: 1100 ms Memory Size: 128 MB Max Memory Used: 128 MB
{"statusCode":200,"body":"{\"listTodo\":[{\"id\":\"7699e070-bbde-4ff3-b2b7-a84bf5d5297a\",\"done\":false,\"title\":\"get coffee\"}]}"}

As you can see I only get single todo back. The get milk todo has been deleted from the database.

View Complete Source code on github

BONUS: Query

As I mentioned earlier the scan operation is the most expensive operation because it will read every single record. And it is not recommended to use scan operation.

So lets say we added authentication to our Todo app and we added rule that users can only access their own todo items and cannot access other peoples todo items.

In this case we are going to create a Global secondary index for owner field.

This way instead of scanning entire table we only read records created by single owner.

So lets add GSI to the table as follows

So we have added new GSI to the table and set simple partition key of type string called owner.

Now you should be able to query the table based on owner. But first we need to add todo items with owner field.

So update existing create todo function as follows

So we have added owner as optional field. This way our function is backwards compatible when we didn't have owner field.

Now let's create query function in the stack as follows

We create function as normal. But main difference here is that when we give this function read access to table then it doesn't include read access to any of its indexes.

Hence we need to create new table resource with existing table name and specify the index name.

Now when you set read access for table with index then it includes read access to all of its indexes.

Without this Important step your query will not run.

Then add query function as follows.

This is very similar to update todo expression. Except in this case the world owner is a reserved keyword.

So we cannot specify it directly. We need to replace it with a variable.

After we make request then rest of it is similar to scan request.

Now let's test it by adding few todo items with owner value of "1212135c-1e2a-44e4-96c4-d41ff9952a4d".

We will add 2 items same as before.

First one is Get milk with events/createWithOwner.json

END RequestId: 7c8a13ff-955b-4796-bccc-ddf62164cfd9
REPORT RequestId: 7c8a13ff-955b-4796-bccc-ddf62164cfd9 Init Duration: 0.10 ms Duration: 1069.35 ms Billed Duration: 1100 ms Memory Size: 128 MB Max Memory Used: 128 MB
{"statusCode":200,"body":"{\"newTodo\":{\"id\":\"9076844c-72f5-4fb1-b26d-e50f557ed9be\",\"owner\":\"1212135c-1e2a-44e4-96c4-d41ff9952a4d\",\"title\":\"get milk\",\"done\":false}}"}

Then the other is Get coffee with events/createWithOwner.json

START RequestId: 28894e07-a3b0-4310-9456-9073522b3cf0 Version: $LATEST
END RequestId: 28894e07-a3b0-4310-9456-9073522b3cf0
REPORT RequestId: 28894e07-a3b0-4310-9456-9073522b3cf0 Init Duration: 0.32 ms Duration: 1062.31 ms Billed Duration: 1100 ms Memory Size: 128 MB Max Memory Used: 128 MB
{"statusCode":200,"body":"{\"newTodo\":{\"id\":\"b40d8163-3e4c-4195-9f55-85500c8b7ad2\",\"owner\":\"1212135c-1e2a-44e4-96c4-d41ff9952a4d\",\"title\":\"get coffee\",\"done\":false}}"}

Now we test query with events/query.json

END RequestId: 1b39370e-b056-4bbc-9174-97a2f46382b0
REPORT RequestId: 1b39370e-b056-4bbc-9174-97a2f46382b0 Init Duration: 0.10 ms Duration: 1039.72 ms Billed Duration: 1100 ms Memory Size: 128 MB Max Memory Used: 128 MB
{"statusCode":200,"body":"{\"listTodo\":[{\"owner\":\"1212135c-1e2a-44e4-96c4-d41ff9952a4d\",\"id\":\"b40d8163-3e4c-4195-9f55-85500c8b7ad2\",\"done\":false,\"title\":\"get coffee\"},{\"owner\":\"1212135c-1e2a-44e4-96c4-d41ff9952a4d\",\"id\":\"9076844c-72f5-4fb1-b26d-e50f557ed9be\",\"done\":false,\"title\":\"get milk\"}]}"}

As you can see with query we got only 2 items with specified owner.

If you scan the table then you will find another item without owner.

So query is more efficient way of getting all the data than scan. So query is highly recommended.

View Complete Source code on github

Conclusion

AWS SDK V3 for javascript has made lambda functions more efficient than before and safe by using typescript.

So I highly recommend that you switch to SDK V3 as soon as possible. And create new lambda functions if not already.