Open in app
Home
Notifications
Lists
Stories

Write
Arockia Nirmal Amala Doss
Arockia Nirmal Amala Doss

Home
About

Sep 14, 2021

Data Engineering Project-AWS Stream and Batch Processing Pipelines for Credit Card Transactions

Some weeks ago I started to build a data engineering project in AWS Cloud. As a Datawarehouse/ETL professional with more than 6+ years of… — Some weeks ago I started to build a data engineering project in AWS Cloud. As a Datawarehouse/ETL professional with more than 6+ years of experience, now I wanted to get into building data warehouses in the cloud. With that in mind and to understand the different cloud services, I built…

AWS

2 min read


Sep 12, 2021

Populating Amazon Redshift DWH from S3 & QuickSight Reporting

In one of previous blog posts here in medium I modeled an OLAP system (Star Schema)for credit card transactions kaggle dataset as below. In… — In one of previous blog posts here in medium I modeled an OLAP system (Star Schema)for credit card transactions kaggle dataset as below. In this blog posts I implemented the OLAP design in Amazon Redshift and built a simple dashboard on the Redshift DWH using Amazon Quicksight.

Redshift

3 min read

Populating Amazon Redshift DWH from S3 & QuickSight Reporting
Populating Amazon Redshift DWH from S3 & QuickSight Reporting

Sep 11, 2021

Amazon QuickSight Dashboard for S3 CSV Data using Redshift Spectrum / Glue Crawler

In this article I would like to write about creating a Amazon QuickSight Dashboard for the data in a csv file located in Amazon S3 sing… — In this article I would like to write about creating a Amazon QuickSight Dashboard for the data in a csv file located in Amazon S3 sing Amazon Redshift Spectrum / Glue Crawler. AWS S3 & GLUE CRAWLER/DATA CATALOG In my last article I wrote about the .csv file I used…

S 3

2 min read

Amazon QuickSight Dashboard for S3 CSV Data using Redshift Spectrum / Glue Crawler
Amazon QuickSight Dashboard for S3 CSV Data using Redshift Spectrum / Glue Crawler

Sep 11, 2021

Amazon QuickSight Dashboard for S3 CSV Data Using Amazon Athena / Glue Crawler

In this article I would like to write about creating a Amazon QuickSight Dashboard for the data in a csv file located in Amazon S3. — In this article I would like to write about creating a Amazon QuickSight Dashboard for the data in a csv file located in Amazon S3. AWS S3 Find and enter S3 from AWS Management Console. Create a S3 bucket “02transactions” with the default options and upload the Transactions.csv …

S 3 Bucket

3 min read

Amazon QuickSight Dashboard for S3 CSV Data Using Amazon Athena / Glue Crawler
Amazon QuickSight Dashboard for S3 CSV Data Using Amazon Athena / Glue Crawler

Aug 16, 2021

Building a Grafana Dashboard in AWS on Serverless Aurora RDS

In my previous article here in Medium, I built a simple stream processing pipeline in AWS which writes credit card transactions data into… — In my previous article here in Medium, I built a simple stream processing pipeline in AWS which writes credit card transactions data into an OLTP Database(Serverless Aurora RDS). https://arockianirmal26.medium.com/building-a-stream-processing-pipeline-in-aws-3224764a6d2b

AWS

4 min read

Building a Grafana Dashboard in AWS on Serverless Aurora RDS
Building a Grafana Dashboard in AWS on Serverless Aurora RDS

Aug 7, 2021

Bulk Import from AWS S3 Bucket into RDS Aurora Serverless using AWS Cloud9

In this article, I am writing about loading the .CSV file data in bulk from AWS S3 bucket into RDS Aurora Serverless using AWS Cloud9. At… — In this article, I am writing about loading the .CSV file data in bulk from AWS S3 bucket into RDS Aurora Serverless using AWS Cloud9. At the time of writing this article, Serverless Aurora clusters don‘t have the feature of loading data from a text file in S3 into the…

AWS

4 min read

Bulk Import from AWS S3 Bucket into RDS Aurora Serverless using AWS Cloud9
Bulk Import from AWS S3 Bucket into RDS Aurora Serverless using AWS Cloud9

Aug 1, 2021

Building a Stream Processing Pipeline in AWS

In my earlier article here in Medium, I took a kaggle credit card transaction dataset and modeled it for the OLTP and OLAP databases as… — In my earlier article here in Medium, I took a kaggle credit card transaction dataset and modeled it for the OLTP and OLAP databases as below. https://arockianirmal26.medium.com/oltp-olap-database-modelling-on-a-kaggle-dataset-3d52c2e02ac7 In this article I am writing, how I created a simple stream processing pipeline in AWS for the OLTP data model as in…

AWS

3 min read

Building a Stream Processing Pipeline in AWS
Building a Stream Processing Pipeline in AWS

Jun 12, 2021

OLTP, OLAP Database Modelling on a Kaggle Dataset

In this article, I took a Kaggle dataset and tried to design data models for OLTP (Online transaction processing) and OLAP (Online… — In this article, I took a Kaggle dataset and tried to design data models for OLTP (Online transaction processing) and OLAP (Online analytical processing) databases. The dataset I chose for this purpose is the credit card transactions which are synthetically generated. It contains the transaction, customer and merchant details. https://www.kaggle.com/kartik2112/fraud-detection

Database Design

2 min read

OLTP, OLAP Database Modelling on a Kaggle Dataset
OLTP, OLAP Database Modelling on a Kaggle Dataset
Arockia Nirmal Amala Doss

Arockia Nirmal Amala Doss

Help

Status

Writers

Blog

Careers

Privacy

Terms

About

Knowable