Local Development Experience Improvements

[desertaxle]

Current behavior

When running a flow without an API URL configured, prefect will start an in-memory ASGI application and use that to handle API calls from the client. This behavior works for HTTP-based API calls but does not work for WebSockets, which are heavily used when working with events. This leads to a bifurcated experience where sometimes the ephemeral API works, but some features only work when sending requests to a persistent server.

Occasionally, a misconfigured profile leads to prefect using of the ephemeral API when it is not intended. The use of prefect does not clearly indicate the common use cases of profiles and when to create and use them. That information is most commonly discovered via the docs.

Proposed behavior

Run the ephemeral server in a subprocess

To resolve the WebSockets not working in ephemeral mode, a server can be started in a subprocess when necessary/requested and shut down when the process that started the server exits. This will likely lead to increased start-up time when in ephemeral mode, but users can use all the same features they would use with a persistent server.

Add an explicit setting for ephemeral mode

Introduce a new setting to explicitly enable starting a subprocess server when necessary to avoid starting a subprocess server unexpectedly. This will ensure connection errors and misconfiguration are reported to the user instead of sending requests to an unintended server.

Add pre-populated profiles for common modes of operation

To show users how profiles can be used in different scenarios, populate a clearly named default set of profiles. The proposed default profiles are as follows:

• ephemeral
  • The default profile when getting started. Will use an ephemeral server by default via the setting proposed above with an on-disk SQLite database.
• local
  • A profile with a configured server address that is templated off the host/port config for the server.
• test
  • A profile to use for development and potentially unit testing. By default, this profile will not persist any information to disk by using an ephemeral server with an in-memory SQLite database and disabling result persistence. This way, users can get a safe local experience when debugging without persisting data alongside other work.
• cloud 
  • A placeholder profile with instructions for authenticating with a Prefect Cloud workspace.