Repository Structure Reorganization

Introduction

This document is intended to promote a constructive discussion on the new structure of pravega-samples repository. This effort aims at achieving two main objectives:

• To improve the accessibility and usability code samples for new users.
• To provide the community with a clear guidelines to contribute new code samples.

Issues: Issue81, Issue83

Preliminary Concepts

As one can easily infer, the complexity and diversity of code samples is potentially large. For this reason, we propose to distinguish between two types of code samples:

• Examples: Examples are small and self-contained code samples intended to demonstrate a particular feature or API related to the component at hand (e.g., Pravega client, Flink connector). As they are supposed to be simple, examples for a component may be packaged and built together to make it easier for users to run them.

• Scenarios: Scenarios are more complex sample applications and their objective is to demonstrate elaborated use-cases (analytics, iot, etc.) using Pravega components. Note that scenarios may have arbitrary dependencies and can be polyglot (i.e., use different Pravega components at the same time). For this reason, scenarios are supposed to be packaged, built and documented as independent sub-projects.

Repository Structure Proposal

We propose to divide the repository into sub-projects (pravega-client-examples, flink-connector-examples and hadoop-connector-examples) addressed to demonstrate a specific component via examples. These packages will pave the way for a first-time user to understand the operation of each component. Also, having the examples decoupled in a per-component basis allows users to focus on the examples of their own interest.

On the other hand, we also include a scenarios folder that contains more complex applications as sub-projects, which show use-cases exploiting one or multiple Pravega components components.

Given the available code samples, the proposal for a the new structure of pravega-samples is as follows:

pravega-samples

    pravega-client-examples

        - gettingstarted
        - consolerw
        - noop
        - statesynchronizer
        - streamcuts

    flink-connector-examples
        - wordcount

    hadoop-connector-examples
        - wordcount

    scenarios
        - turbine-heat-sensor	
        - turbine-heat-processor
        - anomaly-detection