Skip to content

Commit

Permalink
CP 6.0 update and new use case customer object
Browse files Browse the repository at this point in the history
  • Loading branch information
ora0600 committed Dec 9, 2020
1 parent 812ecca commit 4be252c
Show file tree
Hide file tree
Showing 81 changed files with 13,873 additions and 138 deletions.
90 changes: 46 additions & 44 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,77 +1,79 @@
# Confluent Platform 5.5 KSQLDB Hands-on Workshop
# Confluent Platform 6.0.1 KSQLDB Hands-on Workshop
This github project describes a Hands-on Workshop around Confluent KSQLDB. The structure of the Hands-on is as followed
* Explaining and Introduce KSQLDB
* Labs: Get to know the environment
* Advanced explanation of KSQLDB
* Advanced KSQLDB Labs to setup real use cases

* Explaining and Introduce KSQLDB
* Labs: Get to know the environment
* Advanced explanation of KSQLDB
* Advanced KSQLDB Labs to setup real use cases
In general, the hands-on will take 4 hours.

* Start at 10am: Intro and first labs
* break at 12am: 1 hour break
* Continue at 1pm: Additional Labs
* Finish at 3pm
* Start at 10am: Intro and first labs
* break at 12am: 1 hour break
* Continue at 1pm: Additional Labs
* Finish at 3pm

# Environment for Hands-on Workshop Labs
The Hands-on environment can be deployed in three ways

1. run Docker-Compose on your own hardware/laptop use docker-locally
2. create the demo environment in Cloud Provider infrastructure, deploy cloud environment
3. Confluent will deploy a cloud environment for you, and will send you during the workshop all credentials
1. run Docker-Compose on your own hardware/laptop use docker-locally
2. create the demo environment in Cloud Provider infrastructure, deploy cloud environment
3. Confluent will deploy a cloud environment for you, and will send you during the workshop all credentials

# Prerequisites for running environment in Cloud
For an environment in cloud you need to run following components on your machine:

* Browser with internet access
* if you want to deploy in our own environment
* create your own SSH key and deploy in AWS, I use the key name hackathon-temp-key
* terraform have to be installed
* Terraform will install everything you need to execute during the workshop on cloud compute instance
* Having internet access and Ports 8088, 22, 9021 have to be open
* Browser with internet access
* if you want to deploy in our own environment
* create your own SSH key and deploy in AWS, I use the key name hackathon-temp-key
* terraform have to be installed
* Terraform will install everything you need to execute during the workshop on cloud compute instance
* Having internet access and Ports 8088, 22, 9021 have to be open

# Prerequisites for running environment on own hardware
For an environment on your hardware (Macos):
* Docker Desktop installed, give Docker 8GB of your RAM
* curl installed
* jq installed (or don't use jq)
* Docker Desktop installed, give Docker 8GB of your RAM
* curl installed
* jq installed (or don't use jq)

For Windows Users
* as well docker Desktop installed
* Gitbash (Git for windows) installed.
* don't use jq (do not know if this available on Windows)
* as well docker Desktop installed
* Gitbash (Git for windows) installed.
* don't use jq (do not know if this available on Windows)

In general we will work mostly on the prompt, with KSQLDB cli. But you can also use Confluent Control Center KSQLDB GUI is most cases.

# Hands-on Workshop
Before the workshop you will get informed by Confluent with additional information: Access, Webconference dial-in, etc.

* Please prepare yourself
* Please check the documentation and get an rough overview: [ksqldb](https://www.confluent.io/product/ksql/)
* your hardware should be able to run docker-compose. Docker needs at least 8GB of your RAM
* Please prepare yourself
* Please check the documentation and get an rough overview: [ksqldb](https://www.confluent.io/product/ksql/)
* your hardware should be able to run docker-compose. Docker needs at least 8GB of your RAM

Note:
We will ask you before the workshop, if you would like to run on your own environment or if you would like to have an environment provisioned by Confluent in the cloud.

# Hands-on Agenda and Labs:
0. We will start first with an environment check:
* Attendees with an environment provisioned by Confluent should all have an email with credentials etc.
* Is everything up and running: local, cloud or environment giving by confluent.
* [Setup the environment](labs/Setup-Env.md)
* We expect a 20 MIN time-slot for this exercise
* Attendees with an environment provisioned by Confluent should all have an email with credentials etc.
* Is everything up and running: local, cloud or environment giving by confluent.
* [Setup the environment](labs/00_Setup-Env.md)
* We expect a 20 MIN time-slot for this exercise
1. Intro KSQLDB (PPT) - 30 Min
* RECAP KSQLDB - short presentation by presenter (10 minutes)
* What is the structure for today? (20 minutes)
2. Labs Financial service
* [Payment Status Check](labs/usecase_finserv_1.md)
* [Stock price calculation with User defined functions](labs/usecase_finserv_2.md)
* [Create Stocktrade data](labs/usecase_finserv_3.md)
* [Transaction cache](/labs/usecase_finserv_4.md)
3. Labs Retail/Logistics
* [Real-Time Inventory](labs/usecase_realtime_inventory.md)
* [Track & Trace / Shipments](labs/usecase_track-and-trace.md)
* [Distance calculation](labs/usecase_distance.md)
4. Labs Operations
* [Add a KSQLDB App into your environment](labs/ksqldb_operations.md)
* RECAP KSQLDB - short presentation by presenter (10 minutes)
* What is the structure for today? (20 minutes)
2. Labs Financial service - Lab 1 - 4
* [Payment Status Check](labs/01_usecase_finserv_1.md)
* [Stock price calculation with User defined functions](labs/02_usecase_finserv_2.md)
* [Create Stocktrade data](labs/03_usecase_finserv_3.md)
* [Transaction cache](/labs/04_usecase_finserv_4.md)
3. Labs Retail/Logistics - Lab 5 -7
* [Real-Time Inventory](labs/05_usecase_realtime_inventory.md)
* [Track & Trace / Shipments](labs/06_usecase_track-and-trace.md)
* [Distance calculation](labs/07_usecase_distance.md)
4. Labs Customer Object - Lab 8
* [De-Normalize a customer object sourced from a simluated DB](labs/08_ksqldb_customerobject.md)
5. Labs Operations - Lab 9
* [Add a KSQLDB App into your environment](labs/09_ksqldb_operations.md)

We will have a LUNCH Break for 60 Minutes (around 12am) and the workshop will finish around 3pm.

Expand Down
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
@@ -0,0 +1,114 @@
Confluent Community License Agreement
Version 1.0

This Confluent Community License Agreement Version 1.0 (the “Agreement”) sets
forth the terms on which Confluent, Inc. (“Confluent”) makes available certain
software made available by Confluent under this Agreement (the “Software”).  BY
INSTALLING, DOWNLOADING, ACCESSING, USING OR DISTRIBUTING ANY OF THE SOFTWARE,
YOU AGREE TO THE TERMS AND CONDITIONS OF THIS AGREEMENT. IF YOU DO NOT AGREE TO
SUCH TERMS AND CONDITIONS, YOU MUST NOT USE THE SOFTWARE.  IF YOU ARE RECEIVING
THE SOFTWARE ON BEHALF OF A LEGAL ENTITY, YOU REPRESENT AND WARRANT THAT YOU
HAVE THE ACTUAL AUTHORITY TO AGREE TO THE TERMS AND CONDITIONS OF THIS
AGREEMENT ON BEHALF OF SUCH ENTITY. “Licensee” means you, an individual, or
the entity on whose behalf you are receiving the Software.

1. LICENSE GRANT AND CONDITIONS.

1.1 License.  Subject to the terms and conditions of this Agreement,
Confluent hereby grants to Licensee a non-exclusive, royalty-free,
worldwide, non-transferable, non-sublicenseable license during the term
of this Agreement to: (a) use the Software; (b) prepare modifications and
derivative works of the Software; (c) distribute the Software (including
without limitation in source code or object code form); and (d) reproduce
copies of the Software (the “License”).  Licensee is not granted the
right to, and Licensee shall not, exercise the License for an Excluded
Purpose.  For purposes of this Agreement, “Excluded Purpose” means making
available any software-as-a-service, platform-as-a-service,
infrastructure-as-a-service or other similar online service that competes
with Confluent products or services that provide the Software.

1.2 Conditions.  In consideration of the License, Licensee’s distribution
of the Software is subject to the following conditions:

(a) Licensee must cause any Software modified by Licensee to carry
prominent notices stating that Licensee modified the Software.

(b) On each Software copy, Licensee shall reproduce and not remove or
alter all Confluent or third party copyright or other proprietary
notices contained in the Software, and Licensee must provide the
notice below with each copy.  

“This software is made available by Confluent, Inc., under the
terms of the Confluent Community License Agreement, Version 1.0
located at http://www.confluent.io/confluent-community-license.  BY
INSTALLING, DOWNLOADING, ACCESSING, USING OR DISTRIBUTING ANY OF
THE SOFTWARE, YOU AGREE TO THE TERMS OF SUCH LICENSE AGREEMENT.”

1.3 Licensee Modifications.  Licensee may add its own copyright notices
to modifications made by Licensee and may provide additional or different
license terms and conditions for use, reproduction, or distribution of
Licensee’s modifications.  While redistributing the Software or
modifications thereof, Licensee may choose to offer, for a fee or free of
charge, support, warranty, indemnity, or other obligations. Licensee, and
not Confluent, will be responsible for any such obligations.

1.4 No Sublicensing.  The License does not include the right to
sublicense the Software, however, each recipient to which Licensee
provides the Software may exercise the Licenses so long as such recipient
agrees to the terms and conditions of this Agreement.  

2. TERM AND TERMINATION.  This Agreement will continue unless and until
earlier terminated as set forth herein.  If Licensee breaches any of its
conditions or obligations under this Agreement, this Agreement will
terminate automatically and the License will terminate automatically and
permanently.

3. INTELLECTUAL PROPERTY.  As between the parties, Confluent will retain all
right, title, and interest in the Software, and all intellectual property
rights therein.  Confluent hereby reserves all rights not expressly granted
to Licensee in this Agreement. Confluent hereby reserves all rights in its
trademarks and service marks, and no licenses therein are granted in this
Agreement.

4. DISCLAIMER.  CONFLUENT HEREBY DISCLAIMS ANY AND ALL WARRANTIES AND
CONDITIONS, EXPRESS, IMPLIED, STATUTORY, OR OTHERWISE, AND SPECIFICALLY
DISCLAIMS ANY WARRANTY OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR
PURPOSE, WITH RESPECT TO THE SOFTWARE.  

5. LIMITATION OF LIABILITY.  CONFLUENT WILL NOT BE LIABLE FOR ANY DAMAGES OF
ANY KIND, INCLUDING BUT NOT LIMITED TO, LOST PROFITS OR ANY CONSEQUENTIAL,
SPECIAL, INCIDENTAL, INDIRECT, OR DIRECT DAMAGES, HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, ARISING OUT OF THIS AGREEMENT.  THE FOREGOING SHALL
APPLY TO THE EXTENT PERMITTED BY APPLICABLE LAW.

6.GENERAL.

6.1 Governing Law. This Agreement will be governed by and interpreted in
accordance with the laws of the state of California, without reference to
its conflict of laws principles.  If Licensee is located within the
United States, all disputes arising out of this Agreement are subject to
the exclusive jurisdiction of courts located in Santa Clara County,
California. USA.  If Licensee is located outside of the United States,
any dispute, controversy or claim arising out of or relating to this
Agreement will be referred to and finally determined by arbitration in
accordance with the JAMS International Arbitration Rules.  The tribunal
will consist of one arbitrator. The place of arbitration will be Palo
Alto, California. The language to be used in the arbitral proceedings
will be English. Judgment upon the award rendered by the arbitrator may
be entered in any court having jurisdiction thereof.

6.2 Assignment.  Licensee is not authorized to assign its rights under
this Agreement to any third party. Confluent may freely assign its rights
under this Agreement to any third party.

6.3 Other.  This Agreement is the entire agreement between the parties
regarding the subject matter hereof.  No amendment or modification of
this Agreement will be valid or binding upon the parties unless made in
writing and signed by the duly authorized representatives of both
parties.  In the event that any provision, including without limitation
any condition, of this Agreement is held to be unenforceable, this
Agreement and all licenses and rights granted hereunder will immediately
terminate.  Waiver by Confluent of a breach of any provision of this
Agreement or the failure by Confluent to exercise any right hereunder
will not be construed as a waiver of any subsequent breach of that right
or as a waiver of any other right.
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
# Kafka Connect Elasticsearch Connector
[![FOSSA Status](https://app.fossa.io/api/projects/git%2Bhttps%3A%2F%2Fgithub.com%2Fconfluentinc%2Fkafka-connect-elasticsearch.svg?type=shield)](https://app.fossa.io/projects/git%2Bhttps%3A%2F%2Fgithub.com%2Fconfluentinc%2Fkafka-connect-elasticsearch?ref=badge_shield)


kafka-connect-elasticsearch is a [Kafka Connector](http://kafka.apache.org/documentation.html#connect)
for copying data between Kafka and Elasticsearch.

# Development

To build a development version you'll need a recent version of Kafka as well as a set of upstream Confluent projects, which you'll have to build from their appropriate snapshot branch. See the [FAQ](https://github.com/confluentinc/kafka-connect-elasticsearch/wiki/FAQ) for guidance on this process.

You can build kafka-connect-elasticsearch with Maven using the standard lifecycle phases.

# Contribute

- Source Code: https://github.com/confluentinc/kafka-connect-elasticsearch
- Issue Tracker: https://github.com/confluentinc/kafka-connect-elasticsearch/issues


# License

This project is licensed under the [Confluent Community License](LICENSE).


[![FOSSA Status](https://app.fossa.io/api/projects/git%2Bhttps%3A%2F%2Fgithub.com%2Fconfluentinc%2Fkafka-connect-elasticsearch.svg?type=large)](https://app.fossa.io/projects/git%2Bhttps%3A%2F%2Fgithub.com%2Fconfluentinc%2Fkafka-connect-elasticsearch?ref=badge_large)
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
<HTML><HEAD><style>
table {
border-collapse: collapse;
width: 100%;
font-family: sans-serif;
}
h1 {
font-family: sans-serif;
}

th, td {
text-align: left;
padding: 8px;
}
thead td {
background-color: #333;
color: #fff;
font-weight: bold;
}

tr:nth-child(even){background-color: #f2f2f2}

th {
background-color: #000000;
color: white;
}
</style></HEAD><BODY>
<H1> kafka-connect-elasticsearch</H1><br>
<TABLE border="1">
<THEAD>
<TR>
<TD>Artifact</TD><TD>Type</TD><TD>Version</TD><TD>License(s)</TD></TR>
</THEAD>
<TBODY>
<TR>
<TD><A HREF="http://commons.apache.org/proper/commons-codec/">commons-codec-1.9</A></TD><TD>jar</TD><TD>1.9</TD><TD><A HREF="http://www.apache.org/licenses/LICENSE-2.0.txt">Apache 2.0</A><br></TD></TR>
<TR>
<TD><A HREF="http://commons.apache.org/proper/commons-lang/">commons-lang3-3.4</A></TD><TD>jar</TD><TD>3.4</TD><TD><A HREF="http://www.apache.org/licenses/LICENSE-2.0.txt">Apache 2.0</A><br></TD></TR>
<TR>
<TD><A HREF="http://commons.apache.org/proper/commons-logging/">commons-logging-1.2</A></TD><TD>jar</TD><TD>1.2</TD><TD><A HREF="http://www.apache.org/licenses/LICENSE-2.0.txt">Apache 2.0</A><br></TD></TR>
<TR>
<TD>gson-2.4</TD><TD>jar</TD><TD>2.4</TD><TD></TD></TR>
<TR>
<TD>guava-18.0</TD><TD>jar</TD><TD>18.0</TD><TD><A HREF="http://www.apache.org/licenses/LICENSE-2.0.txt">Apache 2.0</A><br></TD></TR>
<TR>
<TD><A HREF="http://hc.apache.org/httpcomponents-asyncclient">httpasyncclient-4.1.1</A></TD><TD>jar</TD><TD>4.1.1</TD><TD><A HREF="licenses/LICENSE-httpasyncclient-4.1.1.txt">included file</A></TD></TR>
<TR>
<TD><A HREF="http://hc.apache.org/httpcomponents-client">httpclient-4.5.1</A></TD><TD>jar</TD><TD>4.5.1</TD><TD><A HREF="licenses/LICENSE-httpclient-4.5.1.txt">included file</A></TD></TR>
<TR>
<TD><A HREF="http://hc.apache.org/httpcomponents-core-ga">httpcore-4.4.4</A></TD><TD>jar</TD><TD>4.4.4</TD><TD><A HREF="licenses/LICENSE-httpcore-4.4.4.txt">included file</A></TD></TR>
<TR>
<TD><A HREF="http://hc.apache.org/httpcomponents-core-ga">httpcore-nio-4.4.4</A></TD><TD>jar</TD><TD>4.4.4</TD><TD><A HREF="licenses/LICENSE-httpcore-nio-4.4.4.txt">included file</A></TD></TR>
<TR>
<TD><A HREF="https://github.com/searchbox-io/Jest">jest-2.0.0</A></TD><TD>jar</TD><TD>2.0.0</TD><TD><A HREF="http://www.apache.org/licenses/LICENSE-2.0.txt">Apache 2.0</A><br></TD></TR>
<TR>
<TD><A HREF="https://github.com/searchbox-io/Jest">jest-common-2.0.0</A></TD><TD>jar</TD><TD>2.0.0</TD><TD><A HREF="http://www.apache.org/licenses/LICENSE-2.0.txt">Apache 2.0</A><br></TD></TR>
<TR>
<TD><A HREF="${confluent.maven.repo}">kafka-connect-elasticsearch-6.0.0-SNAPSHOT</A></TD><TD>jar</TD><TD>6.0.0-SNAPSHOT</TD><TD></TD></TR>
<TR>
<TD><A HREF="http://www.slf4j.org">slf4j-simple-1.7.5</A></TD><TD>jar</TD><TD>1.7.5</TD><TD></TD></TR>
</TBODY>
</TABLE>
</BODY><HTML>
Loading

0 comments on commit 4be252c

Please sign in to comment.