Skip to content
This repository has been archived by the owner on Feb 15, 2024. It is now read-only.

How to read data from Apache Kafka on HDInsight using Spark Structured Streaming.

License

Notifications You must be signed in to change notification settings

Azure-Samples/hdinsight-spark-kafka-structured-streaming

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

20 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

page_type languages products name urlFragment description
sample
java
azure
azure-hdinsight
Use Spark Structured Streaming with Apache Spark and Kafka on HDInsight
hdinsight-spark-kafka-structured-streaming
This example contains a Jupyter notebook that demonstrates how to use Apache Spark structured streaming with Apache Kafka on Azure HDInsight.

Use Spark Structured Streaming with Apache Spark and Kafka on HDInsight

This example contains a Jupyter notebook that demonstrates how to use Apache Spark structured streaming with Apache Kafka on HDInsight.

Note: This example requires Spark 2.2.0, which is available in HDInsight 3.6. The Spark and Kafka clusters must also be in the same Azure Virtual Network. To create a resource group containing all the services needed for this example, use the resource manager template in the Use Spark Structured Streaming with Kafka document.

Contributing

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.