Did you end up solving this in a halfway convenient way?

@bitprophet any comment on the default behavior?

Regarding the reason, comment in the code tells:

                # Take a nap so we're not chewing CPU.
                time.sleep(self.input_sleep)

How to set input_sleep to 0?

A bit surprising one-liner with side-effect

Use current context c.config.runners.local.input_sleep and set it to 0 as in following example:

from invoke import task


@task
def check_config(c):
    c.config.runners.local.input_sleep = 0
    with open("docker-compose.yml", "rb") as in_stream:
        # c.run("docker-compose -f docker-compose.yml config")
        c.run("docker-compose -f - config", in_stream=in_stream)

But be aware: this is changing class (not object) property - so the value holds true for any subsequent call of the Local runner.

In case of tasks.py as follows:

from invoke import task


@task
def check_config(c):
    c.config.runners.local.input_sleep = 0
    with open("docker-compose.yml", "rb") as in_stream:
        # c.run("docker-compose -f docker-compose.yml config")
        c.run("docker-compose -f - config", in_stream=in_stream)

@task
def check_sleepy(c):
    with open("docker-compose.yml", "rb") as in_stream:
        # c.run("docker-compose -f docker-compose.yml config")
        c.run("docker-compose -f - config", in_stream=in_stream)

the timing will depend on order of task executions.

Fast one:

$ invoke check_config check_sleepy

and slow one (as the check_sleepy uses default input_sleep)

$ invoke check_sleepy check_config

For this reason it seems safer to set this on module level, as it is least surprising:

Less surprising method

from invoke.runners import Local

Local.input_sleep = 0

(this is the method @atleta has described)

from invoke import task

from invoke.runners import Local

Local.input_sleep = 0


@task
def check_config(c):
    # c.config.runners.local.input_sleep = 0
    with open("docker-compose.yml", "rb") as in_stream:
        # c.run("docker-compose -f docker-compose.yml config")
        c.run("docker-compose -f - config", in_stream=in_stream)


@task
def check_sleepy(c):
    with open("docker-compose.yml", "rb") as in_stream:
        # c.run("docker-compose -f docker-compose.yml config")
        c.run("docker-compose -f - config", in_stream=in_stream)

It sounds like the right fix for this is to make the sleep only occur when in_stream is not sys.stdin (or if possible some slightly less naive test, but that's all that comes to mind offhand 😐 ). It's definitely required in that case - without it, Invoke and friends use 99% cpu waiting around for the user to type stuff.

But if one is replaying something via a custom input stream that's definitely not needed or, clearly, wanted! I'd accept a patch that does this (usual caveat that it needs tests, etc).

If the purpose of the sleep is to “Take a nap so we're not chewing CPU.” should this sleep be conditional upon there not having been any data in the previous read? (So that we only sleep when waiting for more data, but if there is data, read all of it as fast as possible, and only have one "sleepless-read" each time we reach the end of the available data.)

Also, 1000B buffer seems small – how about something like io.DEFAULT_BUFFER_SIZE (= 8192) as a default for read_chunk_size?

I have encountered this too.

I managed to speed things up a lot by using unittest.mock, but it was still not fast.
Changing the read_chunk_size did make a bit of difference.

I like @asqui's idea that this should only sleep when the buffer is empty.

with (
    mock.patch("time.sleep", new=lambda _: None),
    mock.patch.object(ctx.runners.local, "read_chunk_size", new=io.DEFAULT_BUFFER_SIZE),
):
    ctx.run(...)

I've opened a PR which I hope will address this issue: #983

Nice, thanks for this change!

Is it worth reframing the PR as "Speed up reading from in_stream" and also including the increase of read_chunk_size to io.DEFAULT_BUFFER_SIZE, since you reported that this too made a bit of a difference?

No worries!

Is it worth reframing the PR as "Speed up reading from in_stream"

I'm happy to rename it -- I think that's a better name. Thank you.

and also including the increase of read_chunk_size to io.DEFAULT_BUFFER_SIZE, since you reported that this too made a bit of a difference?

In my case I didn't see any difference once the sleep was avoided. The time went from about a minute to faster than I could blink, and I forgot all about that setting.

In applications with larger buffers I can imagine it making a difference though, so I'll include it anyway!

EDIT: It does indeed make a minor difference in my case, though it's harder to notice because the sleep makes such a huge difference.

@atleta (or anyone else): did you ever figure out how to work around this issue in a Fabric connection?

EDIT: Answering my own question. After creating my Fabric connection (conn), but before using it for anything, I ran this:

conn.config.runners.remote.input_sleep = 0