v0.18.0 : towards multi-protocol
0.18.0
This 0.18.0 release of Smithy4s contains a significant rewrite of the abstractions and the internals that compose the library. It is motivated by a desire to facilitate the catering to two orthogonal use-cases :
- implementation of different protocols.
- integration with other http libraries than the one we provide out-of-the-box.
This release is neither source nor binary compatible with the previous ones, and also forces some users to update their Smithy specifications. Please read the next section carefully.
This release also comes with a number of user-facing improvements.
Behavioural changes
The default timestamp format in Json serialisation is now EPOCH_SECONDS
. This change is motivated by a desire to align with AWS and to improve our compatibility with their tooling. Timestamps shapes (or members pointing to timestamp shapes) now will need to be annotated with @timestampFormat("DATE_TIME")
in order to retrieve the previous behaviour.
Significant rewrite of the abstractions.
The abstractions that power smithy4s have been overhauled to facilitate integration with other protocols than simpleRestJson and other http libraries than http4s. Many levels of the library has been impacted in significant ways, which is likely to break a great many third-party integrations. The amount of breaking changes is too large to list exhaustively, therefore only a highlight is provided in this changelog.
smithy4s.schema.Field
is no longer a GADT differentiating from required/optional fields. There is now asmithy4s.schema.Schema.OptionSchema
GADT member instead, which was required to support some traits.smithy4s.schema.Schema.UnionSchema
now references an ordinal function, as opposed to the previous dispatch function.smithy4s.Endpoint
now contains asmithy4s.schema.OperationSchema
, which is a construct gathering all schemas related to an operation.smithy4s.Endpoint.Middleware
now generalises the concept of "endpoint-specific middleware" to non-http4s centric usecases. It does however loses theMonoid
instance, as it now lives in theinteropcats
package provided by thesmithy4s-cats
module.smithy4s.Service
now allows to get an ordinal value out of a reified operation, thus making it easier to dispatch it to the correct handler.smithy4s.Service
now contains some methods for instantiation of services from an endpoint compilers.- Two new packages in
core
have appeared :smithy4s.server
andsmithy4s.client
, each containing protocol-agnostic constructs that aim at taking care of some of the complexity of integrating libraries/protocols with smithy4s. - A
smithy4s.capability.MonadThrowLike
andsmith4s.capability.Zipper
types have been created, unlocking the writing of generic functions that benefits integrations
with various third-party libraries. smithy4s.http.HttpRequest
andsmithy4s.http.HttpResponse
types have been created.smithy4s.http.HttpUnaryClientCodecs
andsmithy4s.http.HttpUnaryServerCodecs
are new constructs that aim at facilitating the integration of http-protocols. In particular, they take care of a fair amount of complexity related to handlingsmithy.api#http*
traits (including the reconciliation of data coming from http metadata and http bodies).- Overall, the amount of code in the
smithy4s-http4s
module has drastically diminished, as the constructs necessary for the generalisation of the http-related logic have been created. We (maintainers) love http4s, and are not planning on publicly maintaining any other integration, but we are responsible for other integrations in our work ecosystem. Therefore, generalising what we can makes our jobs easier, but also should allow for third parties to have an easier time integrating their http-libraries of choice with Smithy4s.
Highlight : schema partitioning
The most ground-breaking change of 0.18, which is crucial for how things are now implemented, is the addition of a smithy4s.schema.SchemaPartition
utility that allow to split schemas into sub-schemas that each take care of the subset of the data. This mechanism allows to completely decouple the (de)serialisation of http bodies from the decoding of http metadata. This means, for instance, that JSON serialisation no longer has to be aware of traits such as httpHeader
, httpQuery
, httpLabel
. This greatly facilitates the integration of other serialisation technologies (XML, URL Form, etc) as these no longer have to contain convoluted logic related to which fields should be skipped during (de)-serialisation.
As a result, the smithy4s-json module has been rewritten. In particular, the code it contains is now held in the smithy4s.json
package, since it is no longer coupled with http-semantics. The smithy4s.json.Json
object has also been created to provide high-level methods facilitating the encoding/decoding of generated types into json, which is helpful for a number of use-cases that fall out of the server/client bindings.
Features
AWS SDK support.
Smithy4s' coverage of the AWS protocols has increased drastically. Now, the vast majority of services and operations are supported. This does mean that Smithy4s can effectively be used as a cross-platform AWS SDK (with caveats), delegating to http4s
for transport.
The Smithy4s build plugins now also come with utilities to facilitate the code-generation from AWS service specifications.
Please refer yourself to the relevant documentation page.
Build plugins
Smithy has support in IDE via the smithy-language-server. The language server uses a configuration file to understand your project. In 0.18, our build plugins for sbt
and mill
can generate that configuration file for you. Use the following commands depending on the build tool you use, for sbt
: sbt smithy4sUpdateLSPConfig
and for mill
: mill smithy4s.codegen.LSP/updateConfig
.
Mill
The mill
plugin is build for version 0.11.0
. The changes to the API are solely results of this migration.
The most important migration bits:
- Change from
def smithy4sInputDirs: mill.define.Sources
todef smithy4sInputDirs: mill.define.Target[Seq[PathRef]]
- Change from
def manifest: T[mill.modules.Jvm.JarManifest]
todef manifest: T[mill.api.JarManifest]
- Change from
def smithy4sAllExternalDependencies: T[Agg[Dep]]
todef smithy4sAllExternalDependencies: T[Agg[BoundDep]]
Cats Module
Addition of a cats module to contain SchemaVisitor
implementations of commonly-used cats typeclasses. Currently included are cats.Show
and cats.Hash
(note that cats.Eq
is provided by the cats.Hash
implementation).
See #921
Structure Patterns
Allows for marking string types with the alloy#structurePattern
trait. This trait indicates that a given pattern, with parameters, applies to a given string and that this string should
actually be parsed into a structure where the members of the structure are derived from the parameters in the string pattern.
See #942
Non-Orphan Typeclass Instances
Allows creating implicit typeclass instances in the companion objects in the smithy4s-generated code. This is useful for creating instances of
typeclasses such as cats.Show
, cats.Eq
or any others you may be using.
See #912
smithy4s.Blob
smithy4s.ByteArray
has been deprecated in favor of smithy4s.Blob
. This new type is more flexible, in that it can be backed by byte arrays and byte buffers alike.
Additionally, it allows for O(1) concatenation. This change is motivated by a desire to ease integration with third party libraries whilst reducing the need of copies of binary data.
Smithy4s Optics Instances
When the smithy4sRenderOptics setting is enabled, Lenses and Prisms will be rendered in the companion objects of the generated code when appropriate.
See #1103
Open Enumerations
Introduces alternative code generation for enums and intEnums when they are marked with the alloy#openEnum
trait.
See #1137
Union Projections and Visitors
Added convenient methods for working with unions including projectors for each union alternative and a visitor in union companion objects that can be passed to each union's new accept
method.
See #1144
Sparse collections
The sparse
trait is now supported, allowing for the modelling of collections with null values. Its presence leads to the code-generation of List[Option[A]]
and Map[String, Option[A]]
.
See #993
Xml support
The smithy4s-xml
now exists, containing utilities to parse XML blobs into the generated data classes, and render XML from the generated data classes. This serde logic abides by the rules described in the the official smithy documentation.
application/x-www-form-urlencoded support
The smithy4s-core
now contains utilities to parse application/x-www-form-urlencoded payloads into the generated data classes, and render those payloads same payloads the generated data classes. This encoding allows for a few customisation, driven by alloy traits.
See #1113
What's Changed
- Add xml-support [decoder only] by @Baccata in #489
- Add XML support [encoder] by @Baccata in #503
- [WIP] Add AWS Query protocol encoders support by @igor-ramazanov in #700
- Query protocol wiring by @Baccata in #729
- AWS Query protocol : initial setup by @Baccata in #721
- [0.18] Remove CE2 support by @Baccata in #865
- some minor renames/refactors by @kubukoz in #884
- [0.18] Adds the concepts of PartialData/PartialSchema. by @Baccata in #875
- [0.18] Http interpreters rework by @Baccata in #878
- Make decline cli's
service
parameter implicit by @kubukoz in #745 - Dynamic compliance tests by @Baccata in #907
- Adds methods and type aliases to streamline the creation of client interpreters/algebra by @Baccata in #908
- Series/017 into 018 by @kubukoz in #894
- [Aws-Http4s] Fix content header for Aws Signature Client by @yisraelU in #920
- Generate Schema.apply overloads as returning
Schema.StructSchema
by @kubukoz in #919 - [Compliance-tests] Account for TCP routing when testing how routers decode requests. by @Baccata in #922
- Make generated case classes final by @jgoday in #924
- Update http4s-circe, http4s-client, ... to 0.23.18 by @scala-steward in #931
- Add AWS signature tests by @Baccata in #927
- Update fs2-data-xml to 1.7.0 by @scala-steward in #930
- Update jsoniter-scala-core to 2.23.0 by @scala-steward in #929
- Update fs2-core, fs2-io to 3.6.1 by @scala-steward in #928
- [Core] add httpResponseCode support for int enum by @yisraelU in #926
- Update swagger-ui-dist to 4.18.3 by @scala-steward in #936
- Update cats-effect to 3.4.10 by @scala-steward in #938
- Update sbt-sonatype to 3.9.19 by @scala-steward in #941
- Aws Protocol Compliance tests by @yisraelU in #934
- Align
aws-http4s
module on the constructs now provided byhttp4s-kernel
by @Baccata in #914 - Typelevel Cats based module added by @yisraelU in #921
- Update coursier to 2.1.3 by @scala-steward in #947
- Update fs2-data-xml to 1.7.1 by @scala-steward in #950
- start 0.18 changelog by @lewisjkl in #951
- non-orphan implicit typeclasses by @lewisjkl in #912
- Structure patterns by @lewisjkl in #942
- Replace IntEnum hint with enumType field in enumerations by @miguel-vila in #952
- Rework codec interfaces by @Baccata in #953
- [Core : Fix]fix string enum for StringAndBlobCodecSchemaVisitor by @yisraelU in #964
- Update coursier to 2.1.4 by @scala-steward in #979
- Update scala-collection-compat to 2.10.0 by @scala-steward in #935
- Update smithy-build, smithy-model to 1.31.0 by @scala-steward in #939
- Update sbt-sonatype to 3.9.21 by @scala-steward in #985
- Update sbt, sbt-dependency-tree to 1.8.3 by @scala-steward in #983
- Update cats-effect to 3.4.11 by @scala-steward in #984
- Update http4s-circe, http4s-client, ... to 0.23.19 by @scala-steward in #982
- improve: fallback unspecified members of deprecated trait to N/A by @i10416 in #981
- Update jsoniter-scala-core to 2.23.1 by @scala-steward in #978
- Update sbt-dynver to 5.0.1 by @scala-steward in #949
- Update fs2-io to 3.7.0 by @scala-steward in #976
- Update cats-effect to 3.5.0 by @scala-steward in #992
- Query params comparisons (bump alloy), change assertions to use ValidatedNel by @Baccata in #1000
- [Compliance Test] Query Params Comparison by @yisraelU in #959
- fix: avoid baking error metadata into status code by @i10416 in #988
- Add Monoid for http4s middleware by @zetashift in #1002
- misc: update endpoint-middleware.md by @i10416 in #1003
- add option for explicit null encoding 0.18 by @lewisjkl in #996
- Add support for the notion of sparse collection by @Baccata in #993
- ECS provider fix by @msosnicki in #1008
- Adapt to new alloy test configuration format by @Baccata in #1005
- [Json] Handle empty body by @yisraelU in #1001
- [Compliance Tests] extract ComplianceTest weaver based framework by @yisraelU in #937
- Update nscplugin, sbt-scala-native, ... to 0.4.14 by @scala-steward in #1009
- Update sbt-jmh to 0.4.5 by @scala-steward in #1010
- Update alloy-core, alloy-openapi, ... to 0.2.2 by @scala-steward in #1020
- Fixes issues with the build by @daddykotex in #1024
- fix: sync enum trait rendering with enum shape as possible by @i10416 in #980
- Sync aws-rest-json with 0.18 by @Baccata in #1026
- Update http4s-circe, http4s-client, ... to 0.23.20 by @scala-steward in #1032
- GZIP support for aws restJson1 by @daddykotex in #1023
- fix header test by @yisraelU in #1037
- Add support for gzip in aws json by @daddykotex in #1038
- [AWS REST Protocols] Implement AWS header (de)serialisation by @Baccata in #1035
- [AWS SDK] Additional aws encoding details for the Rest Json protocol by @Baccata in #1036
- Support for mill 0.11.0 by @daddykotex in #1025
- Update jsoniter-scala-core to 2.23.2 by @scala-steward in #1043
- Update http4s-circe, http4s-client, ... to 0.23.21 by @scala-steward in #1044
- Backport middleware exception handling changes by @daddykotex in #1042
- Addressing SimpleRestJson tests : by @Baccata in #1040
- AWS-Rest-Json feature branch tweaks by @Baccata in #1045
- [AWS SDK] RestJson1 feature branch by @Baccata in #1018
- 0.17 -> 0.18 sync by @kubukoz in #1041
- Update coursier to 2.1.5 by @scala-steward in #1048
- Move most code-generation to the examples modules by @daddykotex in #1046
- Service Products by @miguel-vila in #1011
- Checksum md5 by @Baccata in #1055
- add md5 checksum middleware by @yisraelU in #1047
- Update sbt-projectmatrix to 0.9.1 by @scala-steward in #1057
- Update http4s-circe, http4s-client, ... to 0.23.22 by @scala-steward in #1059
- Update mill-main, mill-main-api, ... to 0.11.1 by @scala-steward in #1058
- Update sbt-scalajs, scalajs-compiler, ... to 1.13.2 by @scala-steward in #1060
- Update cats-effect to 3.5.1 by @scala-steward in #1065
- Include more compliance tests by @daddykotex in #1061
- Add a version command to the CLI by @daddykotex in #1068
- Revamp of the notion of codec by @Baccata in #1071
- Update smithy-build, smithy-model to 1.33.0 by @scala-steward in #1066
- Add documentation / examples for ServiceProduct usage by @kubukoz in #1054
- Remove remnants of IntEnum by @kubukoz in #1074
- [RestXML Support] Various fixes by @Baccata in #1080
- Overhaul of the
smithy4s.schema.Field
datatype. by @Baccata in #1081 - Implement server endpoint handlers as middlewares by @msosnicki in #1083
- Rework xml decoding to start from an arbitrary XPath by @Baccata in #1085
- Bump @docusaurus/core from 2.3.1 to 2.4.1 in /modules/website by @dependabot in #997
- Update scala-collection-compat to 2.11.0 by @scala-steward in #1039
- RestXml Protocol Support by @lewisjkl in #1079
- Add zlib to flake.nix by @kubukoz in #1090
- escape keywords in namespaces by @lewisjkl in #1087
- Merge series/0.17 into series/0.18 again by @kubukoz in #1089
- Publish snapshots on a schedule, make params bools by @kubukoz in #1093
- Fix snapshot publishing false positives by @kubukoz in #1096
- SimpleRestJson empty optional payloads == null by @Baccata in #1094
- Allow throttled webjars update PRs by @ybasket in #1098
- Bump webjar dependency to 0.47 for 0.18 by @daddykotex in #1099
- Update sbt-mima-plugin to 1.1.3 by @scala-steward in #1106
- Update http4s-circe, http4s-client, ... to 0.23.23 by @scala-steward in #1107
- Minor refactors for 0.18 by @daddykotex in #1105
- smart constructors for union members by @yisraelU in #1109
- Sync with 0.17 (one more time) by @Baccata in #1116
- Update fs2-data-xml to 1.8.0 by @scala-steward in #1095
- Update swagger-ui-dist to 4.19.1 by @scala-steward in #1108
- Ordinal-based coproducts by @Baccata in #1091
- add method to display json shape received when failing document decoder by @yisraelU in #1114
- Add mechanism for granular bloop-config generation by @kubukoz in #1102
- Generate NoStackTrace error types by @yisraelU in #1104
- UrlForm encoding and AWS query client support by @dhpiggott in #1112
- Update sbt-bloop to 1.5.9 by @scala-steward in #1122
- Optics Rendering by @lewisjkl in #1103
- JCodec Allocation Optimization by @lewisjkl in #1125
- use blob in place of bytearray by @lewisjkl in #1115
- Union rendering tweaks by @Baccata in #1126
- Merge 0.17 into 0.18... one more time. by @kubukoz in #1124
- Wire up EC2 Query codec based on Query codec, plus compliance tests by @dhpiggott in #1120
- Render idRefs as smithy4s.ShapeId by @lewisjkl in #1133
- Add ec2Query to supported AWS protocol list by @kubukoz in #1134
- Update fs2-io to 3.8.0 by @scala-steward in #1132
- Update alloy-core, alloy-openapi, ... to 0.2.4 by @scala-steward in #1135
- Fix rendering of IntEnum traits by @Baccata in #1138
- update doc to include smithy4s.meta.noStackTrace usage by @yisraelU in #1140
- Open enumerations by @lewisjkl in #1137
- Update alloy-core, alloy-openapi, ... to 0.2.6 by @scala-steward in #1146
- Add Projections and Visitors to Unions by @lewisjkl in #1144
- add nicer api in new Xml object by @lewisjkl in #1128
- Bump smithy to 1.36.0 by @Baccata in #1150
- Update smithy-build, smithy-model to 1.36.0 by @scala-steward in #1147
- UrlForm decoding support by @dhpiggott in #1113
- Update cats to 2.10.0 by @Baccata in #1156
- Update cats-core to 2.10.0 by @scala-steward in #1154
- Builder classes for service and endpoint by @GaryAghedo in #1097
- [0.18] Fix ClassCastException in document encoding by @kubukoz in #1159
- Include fs2 in root aggregation by @kubukoz in #1167
- remove rendering of required hint as it is redundant by @lewisjkl in #1163
- AWS: set a default timeout on ecs/ec2 cred providers by @kubukoz in #1168
- Update coursier to 2.1.6 by @scala-steward in #1171
- Update jsoniter-scala-core to 2.23.3 by @scala-steward in #1170
- Update smithy-build, smithy-model to 1.37.0 by @scala-steward in #1173
- [0.17] Render extra newline between case class and companion by @kubukoz in #1175
- Update mill-main, mill-main-api, ... to 0.11.2 by @scala-steward in #1178
- Update sbt-bloop to 1.5.10 by @scala-steward in #1177
- Open Enum Transform for AWS by @lewisjkl in #1153
- Update circe-generic, circe-parser to 0.14.6 by @scala-steward in #1181
- Update nscplugin, sbt-scala-native, ... to 0.4.15 by @scala-steward in #1185
- Update sbt-bloop to 1.5.11 by @scala-steward in #1183
- Update jsoniter-scala-core to 2.23.4 by @scala-steward in #1184
- Update slf4j-simple to 2.0.9 by @scala-steward in #1186
- Fix and optimize AWS kernel JS/Native-platform specific code by @armanbilge in #1187
- Update coursier to 2.1.7 by @scala-steward in #1188
- Update sbt-jmh to 0.4.6 by @scala-steward in #1190
- Update mill-main, mill-main-api, ... to 0.11.3 by @scala-steward in #1197
- Revamped Endpoint to use composition instead of inheritance by @Baccata in #1199
- Add narrow/widen/compose polyfunction methods by @Baccata in #1200
- Ergnomics improvements by @Baccata in #1201
- Change matchpath interface to be more flexible by @Baccata in #1204
- Revamped
Errorable
by @Baccata in #1205 - Update smithy-build, smithy-model to 1.38.0 by @scala-steward in #1196
- Update sbt-scalafix to 0.11.1 by @scala-steward in #1191
- Update fs2-core, fs2-io to 3.9.2 by @scala-steward in #1194
- Update swagger-ui-dist to 5.3.2 by @scala-steward in #1162
- Large rewrite of the internals by @Baccata in #1179
- Update mill-main, mill-main-api, ... to 0.11.4 by @scala-steward in #1207
- Rename Readers to Decoders by @Baccata in #1206
- Split Writer and Encoder into two different constructs by @Baccata in #1210
- Fix for Json Cursor by @lewisjkl in #1209
- Add constraints remover transformer for AWS specs by @zetashift in #1166
- Sync 0.18 with 0.17 (late August) by @kubukoz in #1174
- Bump docusaurus-lunr-search from 2.3.2 to 2.4.1 in /modules/website by @dependabot in #1182
- Update jsoniter-scala-core to 2.23.5 by @scala-steward in #1214
- Endpoint Host Prefix by @lewisjkl in #1213
- Update smithy-build, smithy-model to 1.39.0 by @scala-steward in #1217
- Update sbt-scalajs, scalajs-compiler, ... to 1.14.0 by @scala-steward in #1215
- Update swagger-ui-dist to 5.7.2 by @scala-steward in #1216
- Rewrite Blob to facilitate interop with other libs by @Baccata in #1211
- Update cats-effect to 3.5.2 by @scala-steward in #1221
- Attempt to compute the original url, otherwise keep behaviour by @daddykotex in #1220
- Implement the Default shape visitor by @daddykotex in #1223
- Change the DynamicSchemaIndex API by @Baccata in #1222
- branch sync 29 09 2023 by @kubukoz in #1224
- Generate smithy-build.json from sbt config by @daddykotex in #1208
- Bump actions/checkout from 3 to 4 by @dependabot in #1226
- Bump docusaurus-lunr-search from 2.4.1 to 3.0.0 in /modules/website by @dependabot in #1229
- Bump @docusaurus/preset-classic from 2.4.1 to 2.4.3 in /modules/website by @dependabot in #1230
- Bump @docusaurus/core from 2.4.1 to 2.4.3 in /modules/website by @dependabot in #1228
- 0.18 changelog progress by @Baccata in #1212
- Facilitate the loading of AWS specifications by @Baccata in #1219
- Tweak codegen api by @daddykotex in #1225
- Add changelog entry for editor support:smithy-build.json by @daddykotex in #1234
- Update jsoniter-scala-macros to 2.23.5 by @scala-steward in #1235
- Update smithy-build, smithy-model to 1.39.1 by @scala-steward in #1237
- Keep protocol in smithy4s->http4s URI conversion by @kubukoz in #1233
- Update mill-main, mill-main-api, ... to 0.11.5 by @scala-steward in #1239
- Misc tweaks by @Baccata in #1231
- Update upickle to 3.1.3 by @scala-steward in #1236
- Update copyrights by @Baccata in #1240
New Contributors
- @igor-ramazanov made their first contribution in #700
- @jgoday made their first contribution in #924
- @i10416 made their first contribution in #981
- @ybasket made their first contribution in #1098
- @dhpiggott made their first contribution in #1112
- @GaryAghedo made their first contribution in #1097
- @armanbilge made their first contribution in #1187
Full Changelog: v0.17.19...v0.18.0