TABLE OF CONTENTS
In Chapter 1, we ran through the core concepts surrounding the dynamoDb and I provided a link on how to install it locally on your computer.
In Chapter 2, we wrote python scripts to perform CRUD(Create, Read, Update, Delete) operations locally on a table in our dynamoDB.
For this chapter, here's a list of the things we are going to look at
- Querying data from our products table.
- Using Projection Expressions to grab a subset of attributes for each item.
- Using Key Condition Expressions to grab a set of products with the same partition key
Assumption
I assume you've already read the first 2 chapters. If not, please go back and read them.
I assume you've installed a local copy of the dynamoDB on your computer. If not, please go through this link to do that.
https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/DynamoDBLocal.DownloadingAndRunning.html
With all that out of the way, Let's begin
Creating a products Table
I created a products.json file with dummy product items within. Here's how it looks like.
[ {
"id":1,
"title":"Fjallraven - Foldsack No. 1 Backpack, Fits 15 Laptops",
"price":109.95,
"yearManufactured":1998,
"description":"Your perfect pack for everyday use and walks in the forest. Stash your laptop (up to 15 inches) in the padded sleeve, your everyday",
"category":"men clothing",
"image":"https://fakestoreapi.com/img/81fPKd-2AYL._AC_SL1500_.jpg",
"createdDate":"2021-04-16T12:22:41+00:00"
},
{
"id":2,
"title":"Mens Casual Premium Slim Fit T-Shirts ",
"price":22.3,
"yearManufactured":1990,
"description":"Slim-fitting style, contrast raglan long sleeve, three-button henley placket, light weight & soft fabric for breathable and comfortable wearing. And Solid stitched shirts with round neck made for durability and a great fit for casual fashion wear and diehard baseball fans. The Henley style round neckline includes a three-button placket.",
"category":"men clothing",
"image":"https://fakestoreapi.com/img/71-3HjGNDUL._AC_SY879._SX._UX._SY._UY_.jpg",
"createdDate":"2021-04-16T12:22:41+00:00"
}
]
We are going to create a table called products and then bulk load the products.json items into the table.
Currently, there are 20 items in the product.json file.
Our table would have a composite primary key(A partition key and sort Key).
The partition key is the yearManufactured, and the sort key is createdDate.
Here's a python script on how to create the table
import boto3
def create_products_table(dynamodb=None):
if not dynamodb:
dynamodb = boto3.resource('dynamodb', endpoint_url="http://localhost:8000")
table = dynamodb.create_table(
TableName='products',
KeySchema=[
{
'AttributeName': 'yearManufactured',
'KeyType': 'HASH'
},
{
'AttributeName': 'createdDate',
'KeyType': 'RANGE'
}
],
AttributeDefinitions=[
{
'AttributeName': 'yearManufactured',
'AttributeType': 'N'
},
{
'AttributeName': 'createdDate',
'AttributeType': 'S'
},
],
ProvisionedThroughput={
'ReadCapacityUnits': 10,
'WriteCapacityUnits': 10
}
)
return table
if __name__ == '__main__':
product_table = create_products_table()
print("Table status:", product_table.table_status)
The name of my python script is called CreateTableProducts.py.
Run the script like so.
python3 CreateTableProducts.py
AWS deprecated python 2.7, so all python apps should be run with python3.
Take a look at this migration guide in order to get a full understanding of how to migrate, in case you are still using Python 2.
If the table gets created successfully, you can run the below command to see all tables in your database.
aws dynamodb list-tables
Bulk Load Products into table
We have to load a couple of dummy items into our dynamoDB table, in other to quickly illustrate data manipulation.
Create a file called BulkLoadProductsTable.py and type in the following code
import json
from pprint import pprint
from decimal import Decimal
import boto3
def bulk_load_data(products,dynamodb=None):
if not dynamodb:
dynamodb = boto3.resource('dynamodb',endpoint_url="http://localhost:8000")
table = dynamodb.Table('products')
for product in products:
id = int(product['id'])
title = product['title']
print("item:", id,title)
table.put_item(Item=product)
if __name__=='__main__':
with open ('products.json') as json_file:
products = json.load(json_file,parse_float=Decimal)
bulk_load_data(products)
The products.json file is in the same directory as our python scripts. You can get the file here
https://github.com/trey-rosius/dynamodb3.
Now run the python script.
python3 BulkLoadProductsTable.py
Here's how a fragment of the output in terminal looks like.
Query Table
Retrieve a single product *
In order to retrieve a single product from our table, we need to use the partition key(pk) and sort key(sk) of the product.
Let's retrieve a product with pk=1998 and sk="2021-04-16T12:22:41+00:00"
Here's how the code looks like
from pprint import pprint
import boto3
from botocore.exceptions import ClientError
def get_single_product(yearManufactured,createdDate,dynamodb=None):
if not dynamodb:
dynamodb = boto3.resource('dynamodb', endpoint_url="http://localhost:8000")
table = dynamodb.Table('products')
try:
response = table.get_item(Key={'yearManufactured':yearManufactured,'createdDate':createdDate})
except ClientError as e:
print(e.response['Error']['Message'])
else:
return response['Item']
if __name__=='__main__':
product = get_single_product(1998,"2021-04-16T12:22:41+00:00")
if product:
print("successfully retrieved product")
pprint(product,sort_dicts=False)
Here's how you run it
python3 GetSingleProduct.py
Here's the output in terminal
{'image': 'https://fakestoreapi.com/img/71li-ujtlUL._AC_UX679_.jpg',
'createdDate': '2021-04-16T12:22:41+00:00',
'price': Decimal('55.99'),
'description': 'great outerwear jackets for Spring/Autumn/Winter, suitable '
'for many occasions, such as working, hiking, camping, '
'mountain/rock climbing, cycling, traveling or other outdoors. '
'Good gift choice for you or your family member. A warm '
'hearted love to Father, husband or son in this thanksgiving '
'or Christmas Day.',
'id': Decimal('3'),
'title': 'Mens Cotton Jacket',
'category': 'men clothing',
'yearManufactured': Decimal('1998')}
All the attributes for the product have been retrieved. By default, that's how amazon DynamoDB works.
But what if we didn't want to retrieve all the attributes.What if we wanted just id and price of the product? Here's where the projection expression comes in.
Projection expression
When you do a getItem, Query, or scan, amazon DynamoDb returns all item attributes by default. A projection expression is used to get only some, rather than all.
Let's retrieve only the id, price, and category attributes for a given product with partition key as 1998 and sort key as "2021-04-16T12:22:41+00:00"
from pprint import pprint
import boto3
from botocore.exceptions import ClientError
def get_product_with_projection(yearManufactured,createdDate,dynamodb=None):
if not dynamodb:
dynamodb = boto3.resource('dynamodb', endpoint_url="http://localhost:8000")
table = dynamodb.Table('products')
try:
response = table.get_item(Key={'yearManufactured':yearManufactured,'createdDate':createdDate},
ProjectionExpression="id,price,category")
except ClientError as e:
print(e.response['Error']['Message'])
else:
return response['Item']
if __name__=='__main__':
product = get_product_with_projection(1998,"2021-04-16T12:22:41+00:00")
if product:
print("successfully retrieved product")
pprint(product,sort_dicts=False)
Run it
python3 GetSingleProductWithProjection.py
Here's the terminal output.
successfully retrieved product
{'category': 'men clothing', 'price': Decimal('55.99'), 'id': Decimal('3')}
Also, you might want to display a list of categories in the store. Projection expression would come in really handy in this scenario
Key Conditions Expression
Suppose we wanted to retrieve several items with the same partition key from the products table. We could use a Query request with a KeyConditions parameter.
For Example,
Let's get all product id, titles, descriptions, and category, manufactured in the year 2001.
from pprint import pprint
import boto3
from boto3.dynamodb.conditions import Key
def query_and_project_products(yearManufactured, dynamodb=None):
if not dynamodb:
dynamodb = boto3.resource('dynamodb', endpoint_url="http://localhost:8000")
table = dynamodb.Table('products')
print(f"Get id, description, price, and category")
response = table.query(
ProjectionExpression="id, title, description, category",
KeyConditionExpression= Key('yearManufactured').eq(yearManufactured)
)
return response['Items']
if __name__ == '__main__':
yearManufactured = 2001
products = query_and_project_products(yearManufactured)
for product in products:
pprint(product)
Run it
python3 QueryProducts.py
Here's the output
{'category': 'women clothing',
'description': '100% POLYURETHANE(shell) 100% POLYESTER(lining) 75% POLYESTER '
'25% COTTON (SWEATER), Faux leather material for style and '
'comfort / 2 pockets of front, 2-For-One Hooded denim style '
'faux leather jacket, Button detail on waist / Detail '
'stitching at sides, HAND WASH ONLY / DO NOT BLEACH / LINE DRY '
'/ DO NOT IRON',
'id': Decimal('16'),
'title': "Lock and Love Women's Removable Hooded Faux Leather Moto Biker "
'Jacket'}
{'category': 'women clothing',
'description': 'Note:The Jackets is US standard size, Please choose size as '
'your usual wear Material: 100% Polyester; Detachable Liner '
'Fabric: Warm Fleece. Detachable Functional Liner: Skin '
'Friendly, Lightweigt and Warm.Stand Collar Liner jacket, keep '
'you warm in cold weather. Zippered Pockets: 2 Zippered Hand '
'Pockets, 2 Zippered Pockets on Chest (enough to keep cards or '
'keys)and 1 Hidden Pocket Inside.Zippered Hand Pockets and '
'Hidden Pocket keep your things secure. Humanized Design: '
'Adjustable and Detachable Hood and Adjustable cuff to prevent '
'the wind and water,for a comfortable fit. 3 in 1 Detachable '
'Design provide more convenience, you can separate the coat '
'and inner as needed, or wear it together. It is suitable for '
'different season and help you adapt to different climates',
'id': Decimal('15'),
'title': "BIYLACLESEN Women's 3-in-1 Snowboard Jacket Winter Coats"}
As always, you can find the complete source code on GitHub.
https://github.com/trey-rosius/dynamodb3
Conclusion
In this article, we looked at different ways to query data from our dynamoDb table using a projection expression and key condition expressions.
The next article covers Global Secondary Index(GSI)
Thanks for taking out the time in your busy schedule to check this out. I really appreciate it. Hope you learned something new
I might have made a mistake somewhere in the article. If you catch anything, please let me know and I'll get to it immediately.
Don't forget to show a brother some love by liking and commenting too.
Happy Coding
Peace
