Skip to content

Commit

Permalink
doc: improve examples doc
Browse files Browse the repository at this point in the history
  • Loading branch information
hussein-awala committed May 5, 2024
1 parent 22a2093 commit 92f787a
Showing 1 changed file with 34 additions and 13 deletions.
47 changes: 34 additions & 13 deletions docs/user-guide/examples.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,16 @@ built with the spark image tool, as described in the
[spark documentation](https://spark.apache.org/docs/latest/running-on-kubernetes.html#docker-images).

### Python

In this example, we use a small PySpark application that takes a parameter `num_points` to calculate the value of Pi:
```python
--8<-- "examples/python/job.py"
```
and a Dockerfile to package the application with the spark image:
```dockerfile
--8<-- "examples/python/Dockerfile"
```

First, build the docker image and push it to a registry accessible by your cluster,
or load it into your cluster's local registry if you're using minikube or kind:
```bash
Expand All @@ -16,17 +26,29 @@ kind load docker-image pyspark-job
# and then push it to that registry
docker push pyspark-job
```
Then, submit the job:
```bash
python examples/python/submit.py
Then, you can submit the job using the python client:
```python
--8<-- "examples/python/submit.py"
```
Or via the bash script:
or using the CLI:
```bash
./examples/python/submit.sh
--8<-- "examples/python/submit.sh"
```

### Java
Same as above, but with the java example:
This example is similar to the previous one, but it's implemented in java (with maven).

pom.xml:
```xml
--8<-- "examples/java/job/pom.xml"
```
the job class (`src/main/java/com/oss_tech/examples/TestJob.java`):
```java
--8<-- "examples/java/job/src/main/java/com/oss_tech/examples/TestJob.java"
```

Similar to the PySpark application, you need to build the docker image and push it to a registry accessible by
your cluster, or load it into your cluster's local registry if you're using minikube or kind:
```bash
docker build -t java-spark-job examples/java

Expand All @@ -38,12 +60,11 @@ kind load docker-image java-spark-job
# and then push it to that registry
docker push java-spark-job
```
Then, submit the job:
```bash
python examples/java/submit.py
Then, submit the job using the python client:
```python
--8<-- "examples/java/submit.py"
```

Or via the bash script:
or using the CLI:
```bash
./examples/java/submit.sh
```
--8<-- "examples/java/submit.sh"
```

0 comments on commit 92f787a

Please sign in to comment.