Home
WSQ - Microsoft Azure Data Engineer Associate (DP-203)

WSQ - Microsoft Azure Data Engineer Associate (DP-203)

The Microsoft Azure Data Engineer Associate course is meticulously designed to equip professionals with the skills to harness the power of Azure for data engineering. Starting with an introduction to data engineering on Azure, learners will explore the foundational aspects of Azure Data Lake Storage Gen2 and Azure Synapse Analytics, understanding how these platforms facilitate data storage, processing, and analytics at scale. Participants will gain hands-on experience with Azure Synapse serverless SQL pools for querying and transforming data, creating lake databases, and securing data effectively.

Advanced sections of the course delve into data engineering with Azure Synapse Apache Spark Pools, working with data warehouses, and implementing a data streaming solution with Azure Stream Analytics. Learners will explore how to build and manage data pipelines, use Spark Notebooks, and engage in hybrid transactional and analytical processing. The curriculum also covers data governance across an enterprise with Microsoft Purview and data engineering with Azure Databricks, providing insights into cataloging data artifacts, managing Power BI assets, and using Delta Lake for data integrity and versioning. By the end of this course, participants will be proficient in identifying and proposing data sources, operating data warehouse systems, processing multiple data streams, and applying data transformation techniques to maintain data integrity, ensuring they are well-positioned to meet the demands of a data-driven business environment.

Learning Outcomes

By end of the course, learners should be able to:

Identify and propose data sources and processing methods, focusing on business requirements and data specifications to optimize data gathering.
Operate and document data warehouse systems, balancing data access optimization with loading, and map relationships among varied data systems.
Process multiple data streams, utilizing system capabilities to address new data challenges and enhance data processing methods.
Apply data transformation techniques to maintain data integrity, focusing on optimizing data accuracy and functionality in line with business needs.

Course Brochure

Download WSQ - Microsoft Azure Data Engineer Associate (DP-203) Brochure

Skills Framework

This course follows the guideline of Data Engineering ICT-DIT-3005-1.1 under ICT Skills Framework

Certification

Two e-certificates will be awarded to trainees who have demonstrated competency in the WSQ assessment and achieved at least 75% attendance.

Statement of Attainment (SOA) issued by WSG. You can download the SOA from here after 3-4 weeks upon completion of the course. Note that the SOA certificate title indicates the TSC title not the course title.
Certification of Completion issued by Tertiary Infotech Pte Ltd issued after the class.

Disclaimer

We are Official Microsoft Learning Partner (Org ID: 5238476). To get the official Microsoft certification, please register your certification exam at Pearson Vue Test Center.

Certification Exam at Pearson Vue

Once you are prepared for the exam, you can register for the certification here. We are Authorised Pearson Vue Testing Center. You can take the certification exam at our test center. Note that the course fee does not include the certification exam fee.

WSQ Funding

WSQ funding is only applicable to Singaporeans and PR. Subject to eligibility, the funding support is subjected to funding caps.

Effective for courses starting from 1 Jan 2024
Full Fee	GST	Nett Fee after Funding (Incl. GST)
		Baseline	MCES / SME
$2,000	$180.00	$1,180.00	$780.00

Baseline: Singaporean/PR age 21 and above
MCES(Mid-Career Enhanced Subsidy): S'porean age 40 & above

Upon registration, we will advise further on how to tap on the WSQ Training Subsidy.

You can pay the nett fee (after the WSQ training subsidy) by the following :

SkillsFuture Enterprise Credit (SFEC)

Eligible Singapore-registered companies can tap on $10000 SFEC to cover out-of-pocket expenses.Click here to submit SkillsFuture Enterprise Credit

SkillsFuture Credit

Eligible Singapore Citizens can use their SkillsFuture Credit to offset course fee payable after funding. Click here for SkillsFuture Credit submission

PSEA

Eligible Singapore Citizens can use their PSEA funds to offset course fee payable after funding.

To check for Post-Secondary Education Account (PSEA) eligibility for this course, Visit SkillsFuture (course code: TGS-2023039344)

Scroll down to “Keyword Tags” to verify for PSEA eligibility.
If there is “PSEA” under keyword tags, the course is eligible for PSEA.

Course Code: TGS-2024042602

Course Booking

The course fee listed below is before subsidy/grant, if applicable. We will apply for the grant and send you the invoice with nett fee.

$2,000.00 (GST-exclusive)

$2,180.00 (GST-inclusive)

Mode of Training

Physical Classroom Training
Synchronised Teaching using ZOOM
On-Site Training (5 pax and above)

Course Date * Course Date

Course Time *

Sponsorship *

Self-Sponsored
Employer-Sponsored

No. of Participants

Name(s) on Certificate *

NRIC (Last 3 Digits and Alphabet)

Additional Message or Emails for Course Notification

* Required Fields

Post-Course Support

We provide free consultation related to the subject matter after the course. Please email your queries to enquiry@tertiaryinfotech.com and we will forward your queries to the subject matter experts and get back to you asap.

Course Cancellation/Reschedule Policy

We reserve the right to cancel or re-schedule the course due to unforeseen circumstances. If the course is cancelled, we will refund 100% to participants.
Note the venue of the training is subject to changes due to class size and availability of the classroom.
Note the minimal class size to start a class is 3 Pax.

Course Details

Topic 1: Introduction to data engineering on Azure

What is data engineering
Important data engineering concepts
Data engineering in Microsoft Azure

Topic 2: Introduction to Azure Data Lake Storage Gen2

Understand Azure Data Lake Storage Gen
Enable Azure Data Lake Storage Gen2 in Azure Storage
Compare Azure Data Lake Store to Azure Blob storage
Understand the stages for processing big data
Use Azure Data Lake Storage Gen2 in data analytics workloads

Topic 3: Introduction to Azure Synapse Analytics

What is Azure Synapse Analytics
How Azure Synapse Analytics works
When to use Azure Synapse Analytics
Exercise - Explore Azure Synapse Analytics

Topic 4: Use Azure Synapse serverless SQL pool to query files in a data lake

Understand Azure Synapse serverless SQL pool capabilities and use cases
Query files using a serverless SQL pool
Create external database objects
Exercise - Query files using a serverless SQL pool

Topic 5: Use Azure Synapse serverless SQL pools to transform data in a data lake

Transform data files with the CREATE EXTERNAL TABLE AS SELECT statement
Encapsulate data transformations in a stored procedure
Include a data transformation stored procedure in a pipeline
Exercise - Transform files using a serverless SQL pool

Topic 6: Create a lake database in Azure Synapse Analytics

Understand lake database concepts
Explore database templates
Create a lake database
Use a lake database
Exercise - Analyze data in a lake database

Topic 7: Secure data and manage users in Azure Synapse serverless SQL pools

Choose an authentication method in Azure Synapse serverless SQL pools
Manage users in Azure Synapse serverless SQL pools
Manage user permissions in Azure Synapse serverless SQL pools

Topic 8: Analyze data with Apache Spark in Azure Synapse Analytics

Get to know Apache Spark
Use Spark in Azure Synapse Analytics
Analyze data with Spark
Visualize data with Spark
Exercise - Analyze data with Spark

Topic 9: Transform data with Spark in Azure Synapse Analytics

Modify and save dataframes
Partition data files
Transform data with SQL
Exercise: Transform data with Spark in Azure Synapse Analytics

Topic 10: Use Delta Lake in Azure Synapse Analytics

Understand Delta Lake
Create Delta Lake tables
Create catalog tables
Use Delta Lake with streaming data
Use Delta Lake in a SQL pool
Exercise - Use Delta Lake in Azure Synapse Analytics

Topic 11: Analyze data in a relational data warehouse

Design a data warehouse schema
Create data warehouse tables
Load data warehouse tables
Query a data warehouse
Exercise - Explore a data warehouse

Topic 12: Load data into a relational data warehouse

Load staging tables
Load dimension tables
Load time dimension tables
Load slowly changing dimensions
Load fact tables
Perform post load optimization
Exercise - load data into a relational data warehouse

Topic 13: Manage and monitor data warehouse activities in Azure Synapse Analytics

Scale compute resources in Azure Synapse Analytics
Pause compute in Azure Synapse Analytics
Manage workloads in Azure Synapse Analytics
Use Azure Advisor to review recommendations
Use dynamic management views to identify and troubleshoot query performance

Topic 14: Secure a data warehouse in Azure Synapse Analytics

Understand network security options for Azure Synapse Analytics
Configure Conditional Access
Configure authentication
Manage authorization through column and row level security
Exercise - Manage authorization through column and row level security
Manage sensitive data with Dynamic Data Masking
Implement encryption in Azure Synapse Analytics

Topic 15: Build a data pipeline in Azure Synapse Analytics

Understand pipelines in Azure Synapse Analytics
Create a pipeline in Azure Synapse Studio
Define data flows
Run a pipeline
Exercise - Build a data pipeline in Azure Synapse Analytics

Topic 16: Use Spark Notebooks in an Azure Synapse Pipeline

Understand Synapse Notebooks and Pipelines
Use a Synapse notebook activity in a pipeline
Use parameters in a notebook
Exercise - Use an Apache Spark notebook in a pipeline

Topic 17: Plan hybrid transactional and analytical processing using Azure Synapse Analytics

Understand hybrid transactional and analytical processing patterns
Describe Azure Synapse Link

Topic 18: Implement Azure Synapse Link with Azure Cosmos DB

Enable Cosmos DB account to use Azure Synapse Link
Create an analytical store enabled container
Create a linked service for Cosmos DB
Query Cosmos DB data with Spark
Query Cosmos DB with Synapse SQL
Exercise - Implement Azure Synapse Link for Cosmos DB

Topic 19: Implement Azure Synapse Link for SQL

What is Azure Synapse Link for SQL?
Configure Azure Synapse Link for Azure SQL Database
Configure Azure Synapse Link for SQL Server
Exercise - Implement Azure Synapse Link for SQL

Topic 20: Get started with Azure Stream Analytics

Understand data streams
Understand event processing
Understand window functions
Exercise - Get started with Azure Stream Analytics

Topic 21: Ingest streaming data using Azure Stream Analytics and Azure Synapse Analytics

Stream ingestion scenarios
Configure inputs and outputs
Define a query to select, filter, and aggregate data
Run a job to ingest data
Exercise - Ingest streaming data into Azure Synapse Analytics

Topic 22: Visualize real-time data with Azure Stream Analytics and Power BI

Use a Power BI output in Azure Stream Analytics
Create a query for real-time visualization
Create real-time data visualizations in Power BI
Exercise - Create a real-time data visualization

Topic 23: Introduction to Microsoft Purview

What is Microsoft Purview?
How Microsoft Purview works
When to use Microsoft Purview

Topic 24: Discover trusted data using Microsoft Purview

Search for assets
Browse assets
Use assets with Power BI
Integrate with Azure Synapse Analytics

Topic 25: Catalog data artifacts by using Microsoft Purview

Register and scan data
Classify and label data
Search the data catalog

Topic 26: Manage Power BI assets by using Microsoft Purview

Register and scan a Power BI tenant
Search and browse Power BI assets
View Power BI metadata and lineage

Topic 27: Integrate Microsoft Purview and Azure Synapse Analytics

Catalog Azure Synapse Analytics data assets in Microsoft Purview
Connect Microsoft Purview to an Azure Synapse Analytics workspace
Search a Purview catalog in Synapse Studio
Track data lineage in pipelines
Exercise - Integrate Azure Synapse Analytics and Microsoft Purview

Topic 28: Explore Azure Databricks

Get started with Azure Databricks
Identify Azure Databricks workloads
Understand key concepts
Exercise - Explore Azure Databricks

Topic 29: Use Apache Spark in Azure Databricks

Get to know Spark
Create a Spark cluster
Use Spark in notebooks
Use Spark to work with data files
Visualize data
Exercise - Use Spark in Azure Databricks

Topic 30: Use Delta Lake in Azure Databricks

Get Started with Delta Lake
Create Delta Lake tables
Create and query catalog tables
Use Delta Lake for streaming data
Exercise - Use Delta Lake in Azure Databricks

Topic 31: Use SQL Warehouses in Azure Databricks

Get started with SQL Warehouses
Create databases and tables
Create queries and dashboards
Exercise - Use a SQL Warehouse in Azure Databricks

Topic 32: Run Azure Databricks Notebooks with Azure Data Factory

Understand Azure Databricks notebooks and pipelines
Create a linked service for Azure Databricks
Use a Notebook activity in a pipeline
Use parameters in a notebook
Exercise - Run an Azure Databricks Notebook with Azure Data Factory

Final Assessment

Written Assessment (SAQ)
Practical Performance

Course Info

Promotion Code

Promo or discount cannot be applied to WSQ courses

Minimum Entry Requirement

Knowledge and Skills

Able to operate using computer functions with minimum Computer Literacy Level 2 based on ICAS Computer Skills Assessment Framework
Minimum 3 GCE ‘O’ Levels Passes including English or WPL Level 5 (Average of Reading, Listening, Speaking & Writing Scores)

Attitude

Positive Learning Attitude
Enthusiastic Learner

Experience

Minimum of 1 year of working experience.

Target Year Group : 21-65 years old

Minimum Software/Hardware Requirement

Software:

You need to sign up a Azure account (Credit Card is required).

Hardware: Windows and Mac Laptops

About Progressive Wage Model (PWM)

The Progressive Wage Model (PWM) helps to increase wages of workers through upgrading skills and improving productivity.

Employers must ensure that their Singapore citizen and PR workers meet the PWM training requirements of attaining at least 1 Workforce Skills Qualification (WSQ) Statement of Attainment, out of the list of approved WSQ training modules.

For more information on PWM, please visit MOM site.

Funding Eligility Criteria

Individual Sponsored Trainee

Employer Sponsored Trainee

Singapore Citizens or Singapore Permanent Residents
From 1 October 2023, attendance-taking for SkillsFuture Singapore's (SSG) funded courses must be done digitally via the Singpass App. This applies to both physical and synchronous e-learning courses.
Trainee must pass all prescribed tests / assessments and attain 100% competency.
We reserves the right to claw back the funded amount from trainee if he/she did not meet the eligibility criteria.

Singapore Citizens or Singapore Permanent Residents who are DIRECT EMPLOYEE of the sponsoring company.
From 1 October 2023, attendance-taking for SkillsFuture Singapore's (SSG) funded courses must be done digitally via the Singpass App. This applies to both physical and synchronous e-learning courses.
Trainee must pass all prescribed tests / assessments and attain 100% competency.
We reserves the right to claw back the funded amount from the employer if trainee did not meet the eligibility criteria.

SkillsFuture Credit:

Eligible Singapore Citizens can use their SkillsFuture Credit to offset course fee payable after funding.

PSEA:

To check for Post-Secondary Education Account (PSEA) eligibility, goto mySkillsFuture portal and search for this course code.
Scroll down to "Keyword Tags" to verify for PSEA eligibility.
If there is “PSEA” under keyword tags, the course is eligible for PSEA.
And if there is no “PSEA” under keyword tags, the course is ineligible for PSEA.
Not all courses are eligible for PSEA funding.

Absentee Payroll (AP) Funding:

$4.50 per hour, capped at $100,000 per enterprise per calendar year.
AP funding will be computed based on the actual number of training hours attended by the trainee.

SFEC:

If the Training Provider has submitted an enrolment for course fee grant claim in Training Partners Gateway (TPGateway), SSG would be able to derive SFEC funding based on this record. There is no need for enterprise to submit any claim request and the SFEC claim will be automatically generated and disbursed.
Where there is no such record, eligible employers are required to submit an SFEC claim after course completion via the SFEC microsite.
SkillsFuture Enterprise Credit (SFEC) Microsite

Steps to Apply Skills Future Claim

The staff will send you an invoice with the fee breakdown.
Login to the MySkillsFuture portal, select the course you’re enrolling on and enter the course date and schedule.
Enter the course fee payable by you (including GST) and enter the amount of credit to claim.
Upload your invoice and click ‘Submit’

SkillsFuture Level-Up Program

The SkillsFuture Level-Up Programme provides greater structural support for mid-career Singaporeans aged 40 years and above to pursue a substantive skills reboot and stay relevant in a changing economy. For more information, visit SkillsFuture Level-Up Programme

Get Additional Course Fee Support Up to $500 under UTAP

The Union Training Assistance Programme (UTAP) is a training benefit provided to NTUC Union Members with an objective of encouraging them to upgrade with skills training. It is provided to minimize the training cost. If you are a NTUC Union Member then you can get 50% funding (capped at $500 per year) under Union Training Assistance Programme (UTAP).

For more information visit NTUC U Portal – Union Training Assistance Program (UTAP)

Steps to Apply UTAP

Note

SSG subsidy is available for Singapore Citizens, Permanent Residents, and Corporates.
All Singaporeans aged 25 and above can use their SkillsFuture Credit to pay. For more details, visit www.skillsfuture.gov.sg/credit
An unfunded course fee can be claimed via SkillsFuture Credit or paid in cash.
UTAP funding for NTUC Union Members is capped at $250 for 39 years and below and at $500 for 40 years and above.
UTAP support amount will be paid to training provider first and claimed after end of class by learner.

Job Roles

Data Engineer
Azure Data Engineer
Cloud Data Engineer
Business Intelligence Developer
Data Architect
Data Scientist
Machine Learning Engineer
Big Data Engineer
Data Analyst
Database Administrator
Cloud Solutions Architect
DevOps Engineer
Software Developer
IT Professional
Systems Analyst

Trainers

Solomon Soh Zhe Hong: Solomon is ACTA certified and has trained and coached over 100 professionals in the area of data science, python programming and coding. Solomon is a Certified AI Engineer Associate by AI Singapore and holds certifications in Alibaba Cloud Architect and Alteryx respectively. Solomon interests include Reinforcement Learning, Natural Language Processing and Time-Series analysis.

Dr. Alfred Ang: Dr. Alfred Ang is the founder of Tertiary Courses. He is a serial entrepreneur. He founded OSWeb2Design Singapore Pte Ltd in 2007 offering web development, e-commerce store development, graphics design, ebook publishing, mobile apps development, and digital marketing services. He established the first online gardening store in Singapore, Eco City Hydroponics Pte Ltd in 2000, offering a wide range of gardening products such as seeds, plant nutrients, hydroponics kits etc. Eco City Hydroponics has become the most popular and successful gardening store in Singapore. He founded Tertiary Infotech Pte Ltd in 2012 and transformed the business to a training platform, Tertiary Courses in 2014. Tertiary Courses offers a wide range of SkillsFuture courses for PMETs to upgrade their skills and knowledge. He also established Tertiary Courses Malaysia in 2016. He also founded Tertiary Robotics in 2015 offering Arduino, Raspberry Pi, Microbit and Robotics products

Dr. Alfred Ang earned his Ph.D. from National University of Singapore in 2000, majoring in Electrical and Electronics Engineering. He also completed an online MBA course with U21 Global based in Australia. He obtained his B.Sc (Hons) from National University of Singapore in 1992, majoring in Physics. He topped his Physics cohort for 3 consecutive years and funded his degree study with Book price, awards and tuition. He has worked in Defence, Electronics and Semiconductor Industries. His current interests include Machine Learning, Deep Learning, Artificial Intelligence, Internet of Things, Robotics and Programming.

Dr. Alfred Ang is a ACTA certified trainer and DACE certified course developer. He was Distinguished Toastmasters (DTM) and Senior Member of IEEE. He has published more than 20 peer reviewed papers and co-inventors for more than 20 inventions.