Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move to Dockerfile installation #32

Merged
merged 18 commits into from
Aug 31, 2023
Merged
Show file tree
Hide file tree
Changes from 11 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
121 changes: 84 additions & 37 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,77 +2,124 @@

# DDEV-CRON <!-- omit in toc -->

- [Intro](#intro)
- [Introduction](#introduction)
- [Getting started](#getting-started)
- [Implementation](#implementation)
- [./ddev/web-build/cron.conf](#ddevweb-buildcronconf)
- [\*.cron](#cron)
- [Useful sites and debugging](#useful-sites-and-debugging)
- [Examples](#examples)
- [Logging current time](#logging-current-time)
- [TYPO3 scheduler](#typo3-scheduler)
- [Drupal cron](#drupal-cron)
- [Laravel cron](#laravel-cron)

## Intro
## Introduction

This DDEV add-on helps to execute a command in the web container based on a cron schedule. Cron is a classic Linux/Unix service with a well-known configuration syntax.

The add-on
The add-on:

- Installs and runs the cron service inside the web container
- Adds a sample cron configuration that adds to a file every minute.
- Adds an example job that writes out the current time.
- Required DDEV v1.19.3 or higher.
tyler36 marked this conversation as resolved.
Show resolved Hide resolved

*This extension is designed to be a generic implentation. See [Running TYPO3 Cron inside the web container](https://github.com/ddev/ddev-contrib/tree/master/recipes/cronjob) for a specific example of a manual setup.*
*This extension is designed to be a generic implementation. See [Running TYPO3 Cron inside the web container](https://github.com/ddev/ddev-contrib/tree/master/recipes/cronjob) for a specific example of a manual setup.*

## Getting started

- Install the add-on with `ddev get ddev/ddev-cron`
- Update the provided `.ddev/config.cron.yaml` as you see fit with your expected cron jobs (and remove the demonstration line). You can also just add those demonstration lines to your `.ddev/config.yaml` and delete the `.ddev/config.cron.yaml`.
- `ddev restart`
- Install the DDEV cron add-on:

```shell
ddev get ddev/ddev-cron
```

- Customize the cron service, if required, by updating `./.ddev/web-build/cron.conf`
- Remove `#ddev-generated` to prevent DDEV from overriding the file.
tyler36 marked this conversation as resolved.
Show resolved Hide resolved
tyler36 marked this conversation as resolved.
Show resolved Hide resolved
- Add at least one `./ddev/web-build/*.cron` file. This will be automatically added to crontab on startup. See [Implementation](#implementation)
- Restart DDEV to apply changes:

```shell
ddev restart
```
tyler36 marked this conversation as resolved.
Show resolved Hide resolved

## Implementation

The provided `web-build/Dockerfile.ddev-cron` and `web-build/cron.conf` configure the traditional cron daemon to run inside the web container.
This extension does the follow:
tyler36 marked this conversation as resolved.
Show resolved Hide resolved

The `config.cron.yaml` is a simple setup of a trivial cron job within the DDEV web container. It writes a crontab file to configure the cron daemon.
- Adds required cron service to the web container.
- Configures the cron service using `./ddev/web-build/cron.conf`.
- Adds all `./ddev/web-build/*.cron` files to crontab scheduler.

```yaml
hooks:
post-start:
# This adds an every-minute cronjob for your user; it runs "date" and appends that
# to the "time.log" in your project root.
# You can just `ls -l time.log` or `tail -f time.log` to see it happening.
# The crontab can have more than one line for multiple jobs.
# `ddev exec crontab -l` will show you the current crontab configuration
- exec: printf "SHELL=/bin/bash\n* * * * * date >> /var/www/html/time.log\n" | crontab
```
### ./ddev/web-build/cron.conf

This addon provides basic configuration for the cron service out of the box.

To change the configuration for the cron service, update `./ddev/web-build/cron.conf`:

- Remove `#ddev-generated` to prevent DDEV from overriding the file.
- Make any required changes.
tyler36 marked this conversation as resolved.
Show resolved Hide resolved

### *.cron

This addon uses `*.cron` files to populate crontab. This allows projects to track and manage cron jobs via git.

On `ddev start`, all `./ddev/web-build/*.cron` files are:

The default file configures a job to write the date to a log file `time.log` every minute.
It is a simple arbitary example to show the service is working, and remind the user to change it to something more appropriate. You can add additional files into /etc/cron.d, or add additional lines to this one.
- Copied into the `/etc/cron.d`.
- Have their permissions updated.
- Added to crontab.

- If you need help figuring out the syntax of a cron job, see [crontab guru](https://crontab.guru/).
- For the usage of `crontab` see [crontab man page](https://manpages.debian.org/buster/cron/crontab.1.en.html).
- You can experiment with the `crontab` command inside the container by `ddev ssh` and then `crontab -e` for example, or use `ddev exec crontab -e`.
See `.ddev/web-build/time.cron.example` and [Examples](#examples) section below for specific example.

## Useful sites and debugging

- [crontab guru](https://crontab.guru/) is a helpful for generating cron schedule expressions.
- For `crontab` usage, see [crontab man page](https://manpages.debian.org/buster/cron/crontab.1.en.html).
- Check crontab by running `ddev exec crontab -e`.
- If you want the cron to run on your local time instead of UTC, make sure to set `timezone` in your `.ddev/config.yaml`.
- Make sure that when you have tried manually executing the command you want to run inside the container and that it gets the expected results.
- To help debug, connect to the web container session (`ddev ssh`) and manually run the commands to confirm expected results.
- If you are running a CMS command that requires access to the database, set the environment variable `IS_DDEV_PROJECT=true`

## Examples

**TYPO3 scheduler**: A cron to add on to the example and then run the TYPO3 scheduler every minute might be:
The following examples are provide as guides.
PRs are welcome for changes and updates for current best practices for specific frameworks.

```yaml
- exec: printf "SHELL=/bin/bash\n* * * * * date |& tee -a /var/www/html/time.log\n* * * * * IS_DDEV_PROJECT=true /var/www/html/vendor/bin/typo3 scheduler:run -vv |& tee -a /var/www/html/scheduler-log.txt\n" | crontab
### Logging current time

```
This addon provides an example that can check if the cron service is running.
Every minute, it writes the current time (UTC timezone) to `./time.log`.

- Rename `./ddev/web-build/time.cron.example` to `./ddev/web-build/time.cron`
- Restart the DDEV project to start the time example.
- After at least a minute, you should see `./time.log` containing the web container's current time.

### TYPO3 scheduler

See the results of this with `ddev exec crontab -l`:
- Create a `./.ddev/web-build/typo3.cron` file
- Add the following code to run the typo3 scheduler every minute and write to a log file.

```cron
* * * * * cd /var/www/html && IS_DDEV_PROJECT=true vendor/bin/typo3 scheduler:run -vv |& tee -a /var/www/html/scheduler-log.txt
```
SHELL=/bin/bash
* * * * * date |& tee -a /var/www/html/time.log
* * * * * cd /var/www/html && IS_DDEV_PROJECT=true vendor/bin/typo3 scheduler:run -vv |& tee -a /var/www/html/scheduler-log.txt

### Drupal cron

- Create a `./.ddev/web-build/drupal.cron` file
- Add the following code to run the drupal scheduler every 10 minute and write to a log file.

```cron
*/10 * * * * IS_DDEV_PROJECT=true DDEV_PHP_VERSION=8.0 /var/www/html/vendor/bin/drush cron -v |& tee -a /var/www/html/cron-log.txt
```

**Drupal cron**: A cron to run drupal's cron every 10 minutes via drush might be:
### Laravel cron

- Create a `./.ddev/web-build/drupal.cron` file
- Add the following code to run the drupal scheduler minute.

```yaml
- exec: printf "SHELL=/bin/bash\n*/10 * * * * IS_DDEV_PROJECT=true DDEV_PHP_VERSION=8.0 /var/www/html/vendor/bin/drush cron -v |& tee -a /var/www/html/cron-log.txt\n" | crontab
```cron
* * * * * cd /var/www/html && IS_DDEV_PROJECT=true php artisan schedule:run >> /dev/null 2>&1
tyler36 marked this conversation as resolved.
Show resolved Hide resolved
```

**Contributed and maintained by [@tyler36](https://github.com/tyler36) based on the original [Running TYPO3 Cron inside the web container](https://github.com/ddev/ddev-contrib/tree/master/recipes/cronjob) by [@thomaskieslich](https://github.com/thomaskieslich)**
Expand Down
11 changes: 0 additions & 11 deletions config.cron.yaml

This file was deleted.

4 changes: 2 additions & 2 deletions install.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -9,12 +9,12 @@ pre_install_actions:

# list of files and directories listed that are copied into project .ddev directory
project_files:
- config.cron.yaml
- web-build/Dockerfile.ddev-cron
- web-build/cron.conf
- web-build/time.cron.example

# List of files and directories that are copied into the global .ddev directory
global_files:


post_install_actions:

53 changes: 45 additions & 8 deletions tests/test.bats
Original file line number Diff line number Diff line change
Expand Up @@ -20,30 +20,67 @@ teardown() {
[ "${TESTDIR}" != "" ] && rm -rf ${TESTDIR}
}

health_checks() {
# Make sure cron process is running
ddev exec 'sudo killall -0 cron'
}

time_cron_checks() {
# ASSERT time.log was written to
grep UTC time.log

# ASSERT job displays under current user's crontab
ddev exec crontab -l | grep '* * * * * date | tee -a /var/www/html/time.log'
}

@test "install from directory" {
set -eu -o pipefail
cd ${TESTDIR}
echo "# ddev get ${DIR} with project ${PROJNAME} in ${TESTDIR} ($(pwd))" >&3
ddev get ${DIR}
# Set the example cron job as an actual cron job.
mv ./.ddev/web-build/time.cron.example ./.ddev/web-build/time.cron
ddev restart

# The example runs every minute so we should wait at least the length.
sleep 61
# Make sure cron process is running
ddev exec 'sudo killall -0 cron'
# ASSERT: Make sure time.log got a line written to it.
grep UTC time.log

# Check service works
health_checks

# Check example cron job works
time_cron_checks
}

@test "install from release" {
set -eu -o pipefail
cd ${TESTDIR} || ( printf "unable to cd to ${TESTDIR}\n" && exit 1 )
echo "# ddev get ddev/ddev-cron with project ${PROJNAME} in ${TESTDIR} ($(pwd))" >&3
ddev get ddev/ddev-cron
# Set the example cron job as an actual cron job.
mv ./.ddev/web-build/time.cron.example ./.ddev/web-build/time.cron
ddev restart

# The example runs every minute so we should wait at least the length.
sleep 61
# Make sure cron process is running
ddev exec 'sudo killall -0 cron'
# ASSERT: Make sure time.log got a line written to it.
grep UTC time.log

# Check service works
health_checks

# Check example cron job works
time_cron_checks
}

@test "services work when no valid jobs are present" {
set -eu -o pipefail
cd ${TESTDIR}
echo "# ddev get ${DIR} with project ${PROJNAME} in ${TESTDIR} ($(pwd))" >&3
ddev get ${DIR}
ddev restart

# We should wait at least one cycle.
sleep 61

# Check service works
health_checks
}
23 changes: 13 additions & 10 deletions web-build/Dockerfile.ddev-cron
Original file line number Diff line number Diff line change
@@ -1,15 +1,18 @@
#ddev-generated
# Install cron package; this can be done in webimage_extra_packages, but put it here for now.
RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y -o Dpkg::Options::="--force-confold" --no-install-recommends --no-install-suggests cron
# Tell supervisord to start cron service in cron.conf
RUN echo " \n \
[program:cron] \n \
command=sudo /usr/sbin/cron -f -L7 \n \
autorestart=true \n \
startretries=10 \n \
stdout_logfile=/proc/self/fd/2 \n \
stdout_logfile_maxbytes=0 \n \
redirect_stderr=true \n \
" > /etc/supervisor/conf.d/cron.conf

# Copy our custom config
COPY ./cron.conf /etc/supervisor/conf.d/cron.conf

# Make it so you can add to cron.d without root privileges
RUN chmod 777 /etc/cron.d /var/run

# Copy over our custom jobs
COPY ./*.cron /etc/cron.d/*.cron

# Give execution rights on the cron jobs
RUN chmod -f 0644 /etc/cron.d/*.cron || true

# Concatenate files
RUN { cat /etc/cron.d/*.cron; } | crontab -u ${username} -
8 changes: 8 additions & 0 deletions web-build/cron.conf
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
#ddev-generated
[program:cron]
command=sudo /usr/sbin/cron -f -L7
autorestart=true
startretries=10
stdout_logfile=/proc/self/fd/2
stdout_logfile_maxbytes=0
redirect_stderr=true
2 changes: 2 additions & 0 deletions web-build/time.cron.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
#ddev-generated
* * * * * date | tee -a /var/www/html/time.log
Loading