Allow handlers to return user-defined error types #1180

hawkw · 2024-11-20T22:46:29Z

Currently, all endpoint handler functions must return a
Result<T, HttpError>, where T implements HttpResponse. This is
unfortunate, as it limits how error return values are represented in
the API. There isn't presently a mechanism for an endpoint to return
a structured error value of its own which is part of the OpenAPI spec
for that endpoint. This is discussed at length in issue #39.

This branch relaxes this requirement, and instead allows endpoint
handler functions to return any type that implements HttpResponse.
This includes Result<T, HttpError> where T: HttpResponse, but it also
includes any Result<T, E> where T implements HttpResponse and E
implements a new HttpResponseError trait, which defines how to convert
a value into an error response.

Incidentally, changing the return type of handler functions from
Result<T: HttpResponse, HttpError> to just any T: HttpResponse also
permits endpoint handler functions to be infallible and just always
return a response. This was not a primary goal of this change, but it
seems kind of nice to have anyway.

The HttpResponseError trait defines how to produce an error response
for an error value. This is implemented by dropshot's HttpError
type, but it may also be implemented by user errors. Types implementing
this trait must implement HttpResponseContent, to determine how to
generate the response body and define its schema, and they must also
implement a method HttpResponseError::status_code to provide the
status code to use for the error response. This is somewhat different
from the existing HttpCodedResponse trait, which allows successful
responses to indicate at compile time that they will always have a
particular status code, such as 201 Created. Errors are a bit different:
we would like to be able to return any number of different error status
codes, but would still like to ensure that they represent errors, in
order to correctly generate an OpenAPI document where the error schemas
are returned only for error responses (see this comment for
details). As discussed here, we ensure this by providing new
ErrorStatusCode and ClientErrorStatusCode types, which are newtypes
around http::StatusCode that only contain a 4xx or 5xx status (in the
case of ErrorStatusCode), or only contain a 4xx (in the case of
ClientErrorStatusCode). These types may be fallibly converted from an
http::StatusCode at runtime, but we also provide constants for
well-known 4xx and 5xx statuses, which can be used infallibly. The
HttpResponseError::status_code method returns an ErrorStatusCode
rather than a http::StatusCode, allowing us to ensure that error types
always have error statuses and generate a correct OpenAPI document.

Additionally, while adding ErrorStatusCodes, I've gone ahead and
changed the dropshot::HttpError type to also use it, and changed the
HttpError::for_client_error and HttpError::for_status constructors
to take a ClientErrorStatusCode. Although this is a breaking change,
it resolves a long-standing issue with these APIs: currently, they
assert that the provided status code is a 4xx internally, which is often
surprising to the user. Thus, this PR fixes #693.

Fixes #700
Fixes #39
Fixes #693
Fixes #801

This branch is a second attempt at the change originally proposed in PR
#1164, so this closes #1164. This design is substantially simpler, and
only addresses the ability for handler functions to return
user-defined types. Other changes made in #1164, such as a way to
specify a global handler for dropshot-generated errors, and adding
headers to HttpError responses, can be addressed separately. For now,
all extractors and internal errors still produce dropshot::HttpErrors.
A subsequent change will implement a mechanism for providing alternate
presentation for such errors (such as an HTML 404 page).

Enforce that `HttpResponseError`s status codes are 4xx or 5xx using an `ErrorStatusCode` type which may only be those statuses. See: #39 (comment) While we're making breaking API changes, let's also have `HttpError::for_status` take a validated client-error-only type, rather than panicking surprisingly. This way, it's obvious to the user that the argument to this has to be a 4xx. Fixes #693

.envrc

hawkw · 2024-11-21T00:34:17Z

it occurs to me that HttpResponseHeaders might want to forward an implementation of HttpResponseError when the wrapped type implements that. I'll add one.

ahl

thanks for doing this

dropshot/examples/custom-error.rs

ahl · 2024-11-21T00:27:09Z

dropshot/src/api_description.rs

@@ -919,27 +925,59 @@ impl<Context: ServerContext> ApiDescription<Context> {
                }
            };

+            // If the endpoint defines an error type, emit that for


Alternatively, we could have added to components.responses as before and then referenced that. I can see the inline approach you've taken as potentially simpler, though it does bloat up the json output...

So, I'd like to put them in components.responses, too. The reason I didn't is that it might be a bit annoying to determine the name for each response schema. schemars internally disambiguates colliding schema names by turning subsequent ones into like Error2 or whatever, but (AFAICT) we only get that when we actually generate the schema and it gives us back a reference (into components.schemas). We could then try to parse that reference and get the name back out to then use it to generate a components.responses entry for that response, which seems possible, I just thought it seemed annoying enough that I didn't really want to bother with it. Do you think it's worth doing?

What if we name the response based on <T as JsonSchema>::schema_name()? Might that work?

Do I think it's worth doing? I think it's worth trying. It might make the code worse, but it might make the output simpler. At a minimum it will make the diffs against current json simpler. These together--I think--at least warrant giving it a shot.

I don't believe that deduplication is applied to JsonSchema::schema_name(); as
far as I can tell, it only happens once a schema has already been generated,
because that's when the generator can check if the name already exists in the
set of schemas that have been generated so far?

dropshot/src/error.rs

dropshot/src/router.rs

ahl · 2024-11-21T17:42:49Z

dropshot/src/server.rs

@@ -943,9 +946,9 @@ async fn http_request_handle<C: ServerContext>(
                        ),


should we be firing a USDT probe here?

We weren't previously, but yeah, we probably should. I can add it in this PR if that makes sense?

Probably doesn't... (unless you've done it already). We can file an issue

dropshot/tests/fail/bad_endpoint10.stderr

dropshot/tests/fail/bad_endpoint12.stderr

dropshot_endpoint/src/endpoint.rs

Co-authored-by: Adam Leventhal <[email protected]>

dropshot/tests/fail/bad_endpoint10.stderr

ahl · 2024-11-22T17:50:08Z

dropshot/tests/fail/bad_endpoint12.stderr

+   = note: `HttpResponse` is implemented for `Result<T, E>` where `T: HttpResponse` and `E: HttpResponseError`
+   = note: `HttpResponse` is implemented for `http::Response<dropshot::Body>`
+   = help: the trait `HttpResponse` is implemented for `Result<T, E>`
+note: required by a bound in `validate_response_type`


this still feels a little more opaque than bad_endpoint10 -- would this be impacted if we put a #[diagnostic] on HttpResponse? In particular, I'd love to have it be clearer that the problem is String here...

ahl · 2024-11-22T18:00:39Z

dropshot/src/handler.rs

+/// 3. The response returned by the handler function fails to be converted into
+///    a `Response<Body>`.
+///
+/// In cases (1) and (3), the error will always be an [`HttpError`], but in case


Oof! As discussed in DMs, this is kind of awful. In particular it means that sometimes clients will see the custom error type and sometimes not! It's also a mess with regard to OpenAPI. We could just say "the error is oneOf these types", and that's awful too!

As discussed, we might add conversion methods to HttpResponseError for cases 1 and 3 here.

dropshot/tests/fail/bad_trait_endpoint12.stderr

See #1180 (comment) This does mean we can no longer have infallible endpoint handlers, but I never actually wanted that in the first place --- it's just a side benefit.

ahl

I like this direction

ahl · 2024-11-22T22:16:03Z

dropshot/tests/fail/bad_endpoint10.stderr

  --> tests/fail/bad_endpoint10.rs:16:6
   |
 16 | ) -> Result<HttpResponseOk<()>, String> {
-   |      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ the trait `HttpResponse` is not implemented for `Result<HttpResponseOk<()>, String>`
+   |      ^^^^^^ the trait `HttpResponseError` is not implemented for `String`


do we want a #[diagnostic] that helps users find HttpError?

hawkw added 16 commits November 13, 2024 11:22

[WIP] custom error responses using HttpResponse

af226ea

use a new trait, but HttpResponseContent

4d7c3e4

hmmm maybe this is good actually

5bd7e3d

wip schema generation

2eff88a

use schemars existing deduplication

f2c7f5f

just have HttpError be a normal HttpResponseError

86b3afb

just rely on schemars to disambiguate colliding names

1f611bf

start documenting stuff

90e7247

TRYBUILD=overwrite

06a1af3

docs etc

6de6de3

remove unneeded JsonSchema impl for HttpError

cc4a2b9

theory of operation comment in error module

c513c46

actually, we can completely insulate the user from HandlerError

cfc582b

EXPECTORATE=overwrite

3d0575c

fix wsrong doctest

6b4b6d4

hawkw marked this pull request as ready for review November 21, 2024 00:04

Merge branch 'main' into eliza/custom-error-httpresponse-result

5f374b8

hawkw requested review from ahl and davepacheco November 21, 2024 00:05

hawkw added 2 commits November 20, 2024 16:11

rustfmt (NEVER use the github web merge editor)

53ed323

update to track merged changes

ab798a9

ahl reviewed Nov 21, 2024

View reviewed changes

.envrc Outdated Show resolved Hide resolved

EXPECTORATE=overwrite

e87ad82

ahl reviewed Nov 21, 2024

View reviewed changes

hawkw and others added 4 commits November 21, 2024 10:47

Apply docs suggestions from @ahl

6c9c824

Co-authored-by: Adam Leventhal <[email protected]>

remove local envrc

10a4a99

update copyright dates

b9f194c

reticulating comments

576ba5f

hawkw added 4 commits November 21, 2024 13:50

reticulating comments

8a4d52f

nicer error for missing HttpResponse impls

f9642d1

fix trait-based stub API not knowing about error schemas

8f6d70e

EXPECTORATE=overwrite

ccbbbe2

hawkw requested a review from ahl November 21, 2024 22:59

whoops i forgot to add changes to endpoint tests

46b4df1

ahl reviewed Nov 22, 2024

View reviewed changes

convert HttpErrors into endpoint's error type

00bcea7

See #1180 (comment) This does mean we can no longer have infallible endpoint handlers, but I never actually wanted that in the first place --- it's just a side benefit.

ahl reviewed Nov 22, 2024

View reviewed changes

hawkw added 2 commits November 22, 2024 14:59

add a note about HttpError

a6c3472

reticulating implementation comments

4c93e2e

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Allow handlers to return user-defined error types #1180

Allow handlers to return user-defined error types #1180

hawkw commented Nov 20, 2024 •

edited

Loading

hawkw commented Nov 21, 2024

ahl left a comment

ahl Nov 21, 2024

hawkw Nov 21, 2024

ahl Nov 22, 2024

hawkw Nov 22, 2024

ahl Nov 21, 2024

hawkw Nov 21, 2024

ahl Nov 22, 2024

ahl Nov 22, 2024

ahl Nov 22, 2024

ahl left a comment

ahl Nov 22, 2024

		@@ -943,9 +946,9 @@ async fn http_request_handle<C: ServerContext>(
		),

Allow handlers to return user-defined error types #1180

Are you sure you want to change the base?

Allow handlers to return user-defined error types #1180

Conversation

hawkw commented Nov 20, 2024 • edited Loading

hawkw commented Nov 21, 2024

ahl left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ahl left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

hawkw commented Nov 20, 2024 •

edited

Loading