Streaming Data Pipeline with Kafka | Producer Consumer | Schema Registry | Day 2 of 100
Prasoon Parashar Prasoon Parashar
1.31K subscribers
210 views
17

 Published On Aug 18, 2024

LinkedIn:-
Part2 URL:-   • Snowflake Data Pipeline | Setting Up ...  
GitHub :- https://github.com/Noosarpparashar/da...
Kaggle :- https://www.kaggle.com/datasets/ranje...
Instruction Doc :- https://github.com/Noosarpparashar/da...
Snowflake Code :- https://github.com/Noosarpparashar/da...
Code to connect to S3 :- https://github.com/Noosarpparashar/da...
Snowpipe policy:- https://github.com/Noosarpparashar/da...

Introduction (00:00-00:42)
Get Dataset (00:44-01:01)
Set Up Kafka Cluster (01:03-02:11)
Test Kafka by sending message (02:14-03:28)
Send text file data to Topic (03:30-04:58)
Schema registry (05:00-07:05)
Test Schema registry (07:09-07:35)
Consumer Code (07:37-09:59)

show more

Share/Embed