From ab2c69de76f2510b559463cfdfeb853555bee4de Mon Sep 17 00:00:00 2001 From: Nikita Bobko Date: Fri, 9 Aug 2024 19:29:29 +0200 Subject: [PATCH 1/3] KMP Kotlin-to-Java direct actualization --- ...kmp-kotlin-to-java-direct-actualization.md | 211 ++++++++++++++++++ 1 file changed, 211 insertions(+) create mode 100644 proposals/kmp-kotlin-to-java-direct-actualization.md diff --git a/proposals/kmp-kotlin-to-java-direct-actualization.md b/proposals/kmp-kotlin-to-java-direct-actualization.md new file mode 100644 index 000000000..cb41d2b5f --- /dev/null +++ b/proposals/kmp-kotlin-to-java-direct-actualization.md @@ -0,0 +1,211 @@ +# KMP Kotlin-to-Java direct actualization + +* **Type**: Design proposal +* **Author**: Nikita Bobko +* **Contributors**: Dmitriy Novozhilov, Kevin Bierhoff, Mikhail Zarechenskiy, Pavel Kunyavskiy +* **Discussion**: [KEEP-391](https://github.com/Kotlin/KEEP/issues/391) +* **Status**: Submitted +* **Related YouTrack issue**: [KT-67202](https://youtrack.jetbrains.com/issue/KT-67202) + +**Definition.** ClassId is a class identifier that consists of three independent components: the package where class is declared in, names of all outer classes (if presented), and the name of the class itself. +In Kotlin, it could be represented as the data class: +```kotlin +data class ClassId(val package: String, val outerClasses: List, val className: String) +``` + +Two ClassIds are equal if their respective components are equal. +Example: `ClassId("foo.bar", emptyListOf(), "baz")` and `ClassId("foo", listOf("bar"), "baz")` are different ClassIds + +## Introduction + +In Kotlin, there are **two ways** to write an actual declaration for the existing expect declaration. +You can either write an actual declaration with the same ClassId as its appropriate expect and mark the appropriate declarations with `expect` and `actual` keywords (From now on, we will call such actualizations _direct actualizations_), +or you can use `actual typealias`. + +**The first way.** +_direct actualization_ has a nice property that two declarations share the same ClassId. +It's good because when users move code between common and platform fragments, their imports stay unchanged. +But _direct actualization_ has a "downside" that it doesn't allow declaring actuals in external binaries (jars or klibs). +In other words, expect declaration and its appropriate actual must be located in the same "compilation unit." +[Below](#direct-actualization-forces-expect-and-actual-to-be-in-the-same-compilation-unit) we say why, in fact, it's not a "downside" but a "by design" restriction that reflects the reality. + +**The second way.** +Contrary, `actual typealias` forces users to change the ClassId of the actual declaration. +(An attempt to specify the very same ClassId in the `typealias` target leads to `RECURSIVE_TYPEALIAS_EXPANSION` diagnostic) +But we gain the possibility to declare expect and actual declarations in different "compilation units." + +> [!NOTE] +> Though it's a philosophical question what is "the real actual declaration" in this case. +> Is it the `actual typealias` itself (which is still declared in the same "compilation unit"), or is it the target of the `actual typealias` (which, in fact, can be declared in external jar or klib)? + +| | _Direct actualization_ | `actual typealias` | +|---------------------------------------------------------------------|------------------------|--------------------| +| Do expect and actual share the same ClassId? | Yes | No | +| Can expect and actual be declared in different "compilation units"? | No | Yes | + +While `actual typealias` already allows actualizing Kotlin expect declarations with Java declarations (Informally: Kotlin-to-Java actualization), _direct actualization_ only allows Kotlin-to-Kotlin actualizations. +The idea of this proposal is to support _direct actualization_ for Kotlin-to-Java actualizations. + +## Motivation + +As stated in the [introduction](#introduction), unlike `actual typealias`, _direct actualization_ allows to keep the same ClassIds for common and platform declarations in case of Kotlin-to-Java actualization. + +One popular use case for Kotlin-to-Java actualization is KMP-fying existing Java libraries. +For library authors, the possibility to keep the same ClassIds between common and platform declarations is highly valuable: + +- Since it avoids the creation of two ClassIds that refer to the same object, it avoids the confusion on which ClassId should be used +- It simplifies the migration of client code from the Java library to a KMP version of the same library (no need to replace imports) +- It avoid duplication of potentially entire API surface, which can otherwise become cumbersome +- Later replacing the Java actualization with a Kotlin `actual` class is possible without keeping the previous `actual typealias` in place indefinitely + +## The proposal + +**(1)** Introduce `kotlin.annotations.jvm.KotlinActual` annotation in `kotlin-annotations-jvm.jar` +```java +package kotlin.annotations.jvm; + +@Retention(RetentionPolicy.SOURCE) +@Target({ElementType.TYPE, ElementType.METHOD, ElementType.CONSTRUCTOR}) +public @interface KotlinActual { +} +``` + +- The annotation is intended to be used on Java declarations +- **Any usage** of the annotation in Kotlin will be prohibited (even as a type, even as a reference) +- The annotation in Java will function similarly to the `actual` keyword in Kotlin +- It doesn't make sense to mark the annotation with `@ExperimentalMultiplatform` since `OPT_IN_USAGE_ERROR` is not reported in Java +- `ElementType.FIELD` annotation target is not specified by design. It's not possible to actualize Kotlin properties with Java fields + +**(2)** If Kotlin expect and Java class have the same ClassId, Kotlin compiler should consider Kotlin expect class being actualized with the appropriate Java class. +In other words, support _direct actualization_ for Kotlin-to-Java actualization. + +**(3)** Kotlin compiler should require using `@KotlinActual` on the Java top level class and its respective members. +The rules must be similar to the one with `actual` keyword requirement in Kotlin-to-Kotlin actualization. + +**(4)** If `@KotlinActual` is used on Java class members that don't have respective Kotlin expect members, it should be reported by the Kotlin compiler. +The rules must be similar to the one with `actual` keyword requirement in Kotlin-to-Kotlin actualization. +Please note that Kotlin can only detect excessive `@KotlinActual` annotations on methods of the classes that actualize some existing Kotlin expect classes. +Since Kotlin doesn't traverse all Java files, it's not possible to detect excessive `@KotlinActual` annotation on the top-level Java classes for which a respective Kotlin expect class doesn't exist. +For the same reason, it's not possible to detect excessive `@KotlinActual` annotation on members of such Java classes. +For these cases, it's proposed to implement Java IDE inspection. + +**Worth noting.** +1. Java wasn't capable and still is not capable of actualizing Kotlin top-level functions in any way. +2. Kotlin expect classes are not capable of expressing Java static members [KT-29882](https://youtrack.jetbrains.com/issue/KT-29882) + +Example of a valid Kotlin-to-Java direct actualization: +```kotlin +// MODULE: common +expect class Foo() { + fun foo() +} + +// MODULE: JVM +@kotlin.annotations.jvm.KotlinActual public class Foo { + @kotlin.annotations.jvm.KotlinActual public Foo() {} + @kotlin.annotations.jvm.KotlinActual public void foo() {} + + @Override + public String toString() { return "Foo"; } // No @KotlinActual is required +} +``` + +## actual keyword is a virtue + +An alternative suggestion is to match only by ClassIds, and to drop `actual` keyword in Kotlin-to-Kotlin actualizations, and to drop `@KotlinActual` annotation in Kotlin-to-Java actualization. + +The suggestion was rejected because we consider `actual` keyword being beneficial for readers, much like the `override` keyword. + +- **Misspelling prevention.** + The explicit `actual` keyword (or `@KotlinActual` annotation) helps against misspelling + (esp. when Kotlin supports expect declarations with bodies [KT-20427](https://youtrack.jetbrains.com/issue/KT-20427)) +- **Explicit intent.** + Declarations may be written solely to fullfil the "expect requirement" but the declarations may not be directly used by the platform code. + The `actual` keyword (or `@KotlinActual` annotation) explictly signals the intention to actualize member rather than accidentally defining a new one. + (too bad that members of `actual typealias` break the design in this place) +- **Safeguards during refactoring.** + If the member in expect class changes (e.g. a new parameter added), `actual` keyword (or `@KotlinActual` annotation) helps to quickly identify members in the actual class that needs an update. + Without the keyword, there might be already a suitable overload in the actual class that would silently become a new actualization. + +To support our arguments, we link Swift community disscussions about the explicit keyword for protocol conformance (as of swift 5.9, no keyword is required): +[1](https://forums.swift.org/t/pre-pitch-explicit-protocol-fulfilment-with-the-conformance-keyword/60246), [2](https://forums.swift.org/t/keyword-for-protocol-conformance/3837) + +## The proposal doesn't cover Kotlin-free pure Java library use case + +There are two cases: +1. The user has a Kotlin-Java mixed project, and they want to KMP-fy it. +2. The user has a pure Java project, and they want to KMP-fy it. + +The first case is handled by [the proposal](#the-proposal). + +In the second case, the common part of the project is Kotlin sources that depend on `kotlin-stdlib.jar`. +The common part of the project may also define additional regular non-expect Kotlin declarations. +JVM part of the project depends on common. + +If users want to keep their JVM part free of Kotlin, they have to be accurate and avoid accidental usages of `kotlin-stdlib.jar`, and avoid declaring additional non-expect declarations in the common. +[The proposal](#the-proposal) doesn't cover that case well since the design would become more complicated. +The current proposal is a small incremental addition to the existing model, and it doesn't block us from covering the second case later if needed. + +`KotlinActual` annotation has `SOURCE` retention by design. +This way, the annotation is least invasive for the Java sources, and it should be enough to have compile-only dependency on `kotlin-annotations-jvm.jar`. +Which doesn't contradict the "pure Java project" case. + +## Kotlin-to-Java expect-actual incompatibilities diagnostics reporting + +**Invariant 1.** Kotlin compiler cannot report compilation errors in non-kt files. + +In Kotlin-to-Kotlin actualization, expect-actual incompatibilities are reported on the actual side. + +In Kotlin-to-Java actualization, it's proposed to report incompatibilities on the expect side. +It's inconsistent with Kotlin-to-Kotlin actualizations, but we don't believe that Kotlin-to-Java actualization is significant enough to break the _invariant 1_. +The reporting may be improved in future versions of Kotlin. + +## Direct actualization forces expect and actual to be in the same compilation unit + +In the [introduction](#introduction), we mentioned that _direct actualization_ forces expect and actual to be in the same "compilation unit." +It's an implementation limitation that we believe is beneficial, because it reflects the reality. + +It's a common pattern for libraries to use a unique package prefix. +We want people to stick to that pattern. +_Direct actualization_ for external declarations encourages wrong behavior. + +- Imagine that it is possible to write an expect declaration for the existing JVM library via _direct actualization_ mechanism. + Users may as easily decide to declare non-expect declarations in the same package, which leads to the "split package" problem in JPMS. +- Test case: there is an existing JVM library A. + Library B is a KMP wrapper around library A. + Library B provides expect declarations for the library A. + Later, Library A decides to provide its own KMP API. + If Library B could use _direct actualization_, it would lead to declarations clash between A and B. + +## The frontend technical limitation of Kotlin-to-Java direct actualization + +Frontend transformers are run only on Kotlin sources. +Java sources are visited lazily only if they are referenced from Kotlin sources. + +Given frontend technical restriction, it's proposed to implement Kotlin-to-Java _direct actualization_ matching and checking only on IR backend. + +## Alternatives considered + +- Implicitly match Kotlin-to-Java with the same ClassId if some predefined annotation is presented on the expect declaration. +- `actual typealias` in Kotlin without target in RHS. +- `actual` declaration that doesn't generate class files. + + It could be some special annotation that says that bytecode shouldn't be generated for the class. + The idea is useful by itself, for example, in stdlib, to declare `kotlin.collections.List`. + + The disadvantages are clear: unlike the current proposal, there will be no compilation time checks; + compared to the current proposal, it will result in excessive code duplication in the expect-actual case. + +See [actual keyword is a virtue](#actual-keyword-is-a-virtue) to understand why alternatives were discarded. +Besides, the proposed solution resembles the already familiar Kotlin-to-Kotlin _direct actualization_, but makes it available for Java. + +## Unused declaration inspection in Java + +IntelliJ IDEA implements an unused declaration inspection. +The `javac` itself doesn't emit warnings for unused declarations. + +The inspection in IDEA should be changed to account for declarations annotated with `@kotlin.annotations.jvm.KotlinActual`. + +## Feature interaction with hierarchical multiplatform + +There is no feature interaction. It's not possible to have Java files in intermediate fragments. From 1ef2e4aed5951edd04e96f51c8cf7383d3200920 Mon Sep 17 00:00:00 2001 From: Nikita Bobko Date: Mon, 2 Sep 2024 12:01:16 +0200 Subject: [PATCH 2/3] fixup! KMP Kotlin-to-Java direct actualization --- proposals/kmp-kotlin-to-java-direct-actualization.md | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/proposals/kmp-kotlin-to-java-direct-actualization.md b/proposals/kmp-kotlin-to-java-direct-actualization.md index cb41d2b5f..12fb42736 100644 --- a/proposals/kmp-kotlin-to-java-direct-actualization.md +++ b/proposals/kmp-kotlin-to-java-direct-actualization.md @@ -70,11 +70,12 @@ public @interface KotlinActual { } ``` -- The annotation is intended to be used on Java declarations -- **Any usage** of the annotation in Kotlin will be prohibited (even as a type, even as a reference) -- The annotation in Java will function similarly to the `actual` keyword in Kotlin -- It doesn't make sense to mark the annotation with `@ExperimentalMultiplatform` since `OPT_IN_USAGE_ERROR` is not reported in Java -- `ElementType.FIELD` annotation target is not specified by design. It's not possible to actualize Kotlin properties with Java fields +- The annotation is intended to be used on Java declarations +- **Any usage** of the annotation in Kotlin will be prohibited (even as a type, even as a reference). + As if `KotlinActual` was annotated with `@kotlin.Deprecated(level = DeprecationLevel.ERROR)` +- The annotation in Java will function similarly to the `actual` keyword in Kotlin +- It doesn't make sense to mark the annotation with `@ExperimentalMultiplatform` since `OPT_IN_USAGE_ERROR` is not reported in Java +- `ElementType.FIELD` annotation target is not specified by design. It's not possible to actualize Kotlin properties with Java fields **(2)** If Kotlin expect and Java class have the same ClassId, Kotlin compiler should consider Kotlin expect class being actualized with the appropriate Java class. In other words, support _direct actualization_ for Kotlin-to-Java actualization. From a005e8e3fef1c4e11ac5a45cd5be2f4ea26f1e45 Mon Sep 17 00:00:00 2001 From: Nikita Bobko Date: Mon, 2 Sep 2024 12:01:16 +0200 Subject: [PATCH 3/3] fixup! KMP Kotlin-to-Java direct actualization --- proposals/kmp-kotlin-to-java-direct-actualization.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/proposals/kmp-kotlin-to-java-direct-actualization.md b/proposals/kmp-kotlin-to-java-direct-actualization.md index 12fb42736..8ffaeaac7 100644 --- a/proposals/kmp-kotlin-to-java-direct-actualization.md +++ b/proposals/kmp-kotlin-to-java-direct-actualization.md @@ -14,7 +14,7 @@ data class ClassId(val package: String, val outerClasses: List, val clas ``` Two ClassIds are equal if their respective components are equal. -Example: `ClassId("foo.bar", emptyListOf(), "baz")` and `ClassId("foo", listOf("bar"), "baz")` are different ClassIds +Example: `ClassId("foo.bar", emptyList(), "baz")` and `ClassId("foo", listOf("bar"), "baz")` are different ClassIds ## Introduction @@ -55,7 +55,7 @@ For library authors, the possibility to keep the same ClassIds between common an - Since it avoids the creation of two ClassIds that refer to the same object, it avoids the confusion on which ClassId should be used - It simplifies the migration of client code from the Java library to a KMP version of the same library (no need to replace imports) -- It avoid duplication of potentially entire API surface, which can otherwise become cumbersome +- It avoids duplication of potentially entire API surface, which can otherwise become cumbersome - Later replacing the Java actualization with a Kotlin `actual` class is possible without keeping the previous `actual typealias` in place indefinitely ## The proposal