Significant changes to impl trap in Rust 2024

Rust 2024 versionimpl TraitSignificant adjustments have been made to the default behavior of returning to the location, aimed at simplifying itimpl TraitUse it to better meet the common needs of developers and provide more flexible syntax to achieve finer control.

`impl Trait`Background knowledge

This article discusses the return of locationimpl Trait**For example:

fn process_data(
    data: &[Datum]
) -> impl Iterator<Item = ProcessedDatum> {
    data
        .iter()
        .map(|datum| datum.process())
}

here-> impl IteratorThe function returns a ‘certain type of iterator’ whose actual type is determined by the compiler based on the function body, known as a ‘hidden type’. The caller cannot know the exact type and can only targetIteratorEncode the trait. However, during code generation, the compiler generates code based on the actual type to ensure that the caller obtains complete optimization.

Although the caller does not know the exact type, it needs to know that it will continue to borrowdataParameters to ensure during the iteration processdataThe citation remains valid. In addition, the caller must be able to determine this based solely on the type signature without looking at the function body.

The current rule in Rust is to return the positionimpl TraitValues can only be used when they appearimpl TraitReference to its own lifecycle. In this example,impl Iterator<Item = ProcessedDatum>No lifecycle is referenced, therefore captureddataIt is illegal. You canPlayground[1]Verify in person.

In this case, the error message you receive (“Hidden type captures lifecycle”) is not intuitive, but it does provide a useful repair suggestion:

help: to declare that
      `impl Iterator<Item = ProcessedDatum>`
      captures `'_`, you can add an
      explicit `'_` lifetime bound
  |
5 | ) -> impl Iterator<Item = ProcessedDatum> + '_ {
  |                                           ++++

According to this suggestion, the function signature should be changed to:

fn process_data<'d>(
    data: &'d [Datum]
) -> impl Iterator<Item = ProcessedDatum> + 'd {
    data
        .iter()
        .map(|datum| datum.process())
}

In this version, the data lifecycle'dstayimpl TraitThe type is explicitly referenced, so it can be used. This also indicates to the caller that,dataThe borrowing of must continue until the iterator is used up, which means it will (correctly) mark errors in examples like this（ Try on Playground [2]）：

let mut data: Vec<Datum> = vec![Datum::default()];
let iter = process_data(&data);
data.push(Datum::default());     // <-- !    
iter.next();

Availability issues with existing designs

Regarding which generic parameters can be usedimpl TraitThe rules used in the early stages were determined based on limited examples. Over time, we have noticed some issues.

Default behavior is unreasonable

A survey of the main code repositories (including compilers and crates on Crates. io) found that the vast majority of them return the locationimpl TraitValues require a lifecycle, so not capturing default behavior is not helpful.

Lack of flexibility

The current rule is to return the locationimpl Trait always Allow the use of type parameters, Sometimes Allow the use of lifecycle parameters (if they appear in the boundary). As mentioned above, this default value is incorrect because most functions actually really I hope their return types allow the use of lifecycle parameters: there is at least one solution to this (apart from some details we will explain below). But the default values are also incorrect, because some functions want to explicitly declare them in the return type no Using type parameters, but currently there is no way to override this. The initial idea was Type alias’ impl trait ’ [3]This problem can be solved, but it would be a very ergonomic solution (and due to other complex factors, the stability of the type alias’ impl trait ‘may take longer than expected).

Difficult to explain

Because the default values are incorrect, users often encounter these errors, which are subtle and difficult to explain (as this article proves!). Add compiler prompts to suggest+ '_It’s very helpful, but users have to follow prompts they don’t fully understand, which is not good.

Suggestion incorrect

Add one+ '_Parameters toimpl TraitIt may be confusing, but it’s not difficult. Unfortunately, it is often incorrect comments that can lead to unnecessary compiler errors - and correct The repair process is either complex or sometimes even impossible. Consider this example:

fn process<'c, T> {
    context: &'c Context,
    data: Vec<T>,
) -> impl Iterator<Item = ()> + 'c {
    data
        .into_iter()
        .map(|datum| context.process(datum))
}

hereprocessThe function willcontext.processbe applied todata(Type is)T）Each element in it. Because the return value was usedcontextTherefore, it is declared as+ 'c. Our real goal here is to allow the use of return types'c； to write+ 'cThis goal has been achieved because'cNot appearing in the boundary list. However, although written+ 'cYes, it is'cA convenient way to appear in the boundary, but it also means that the hidden type must be'cLive longer. This requirement is unnecessary, in fact, it can lead to compilation errors in this example（ Try on Playground [4]）。

The reason for this error is a bit subtle. Hidden type is based ondata.into_iter()The iterator type of the result, which will contain the typeT. because+ 'cThe limitation is that hidden types must be compared to'cLiving longer, which in turn meansTMust compare'cLive longer. howeverTIt is a generic parameter, so the compiler needs a parameter likewhere T: 'cSuch a ‘where’ clause. The meaning of this’ where ‘clause is’ create a lifecycle for’'cThe pointing typeTThe citation is safe. But in fact, we did not create any such references, so the ‘where’ clause should not be necessary. It is necessary because we have used convenient but sometimes incorrect solutions, namelyimpl TraitAdd to the boundary+ 'c。

As before, this error is very subtle and touches upon more complex parts of the Rust type system. Unlike before, there is no simple solution! This issue frequently occurs in compilers, leading to a The obscure solution called the ‘Captures’ trait [5]. That’s so disgusting!

We investigated crates on crates.io and found that in the vast majority of cases involving return position impl traits and generics, the boundaries are too strong and may lead to unnecessary errors (although they are often used in simple ways that do not trigger errors).

Inconsistent with other parts of Rust

The current design also introduces inconsistencies with other parts of Rust.

Asynchronous FN Sugar Removal

Rust willasync fnDefined as converting sugar into a return-> impl FutureThe ordinaryfn. Therefore, you may expect something likeprocessThis function:

async fn process(data: &Data) { .. }

… will be (roughly) sugar broken down into:

fn process(
    data: &Data
) -> impl Future<Output = ()> {
    async move {
        ..
    }
}

In practice, due to issues with the rules regarding which lifecycles can be used, this is not a true solution to sugar. The actual process of sugar digestion is a special oneimpl TraitIt allows the use of all lifecycles. But this form ofimpl TraitIt has not been exposed to end-users.

The impl trait in the trait

When we pursue the design of the impl trait in our traits（RFC 3425[6]）We have encountered some challenges related to lifecycle capture. To achieve the symmetry we desire [7]For example, it can be written in a trait-> impl FutureAnd to achieve the expected results, we had to change the rules to allow hidden types to be used uniformly All of them Generic parameters (type and lifecycle).

Rust 2024 Design

The above issues prompted us to adopt a new approach in Rust 2024. This method is a combination of the following two:

A new default value, which returns the locationimpl TraitThe hidden types can be used within the scope whatever Generic parameters, not just types (applicable only to Rust 2024);
A syntax used to explicitly declare which types can be used (in any version).

The new explicit syntax is called ‘use bound’: for example,impl Trait + use<'x, T>Allow hidden types to be used'xAndT(However, the use of any other generic parameters within the scope is not allowed).

The lifecycle can now be used by default

In Rust 2024, by default, returns the positionimpl TraitThe hidden types of values are used within the scope whatever Generic parameters, whether they are types or lifecycles. This means that the initial example of this blog post can be well compiled in Rust 2024（ Try it yourself by setting the version in Playground to 2024 [8]）：

fn process_data(
    data: &[Datum]
) -> impl Iterator<Item = ProcessedDatum> {
    data
        .iter()
        .map(|datum| datum.process())
}

That is great!

Impl Trap can contain a`use<>`Boundaries to precisely specify which generic types and lifecycles they use

As a side effect of this change, if you manually move the code to Rust 2024 (without using)cargo fix）It may be possible to haveimpl TraitThe caller of the return type function is starting to receive errors. This is because assuming theseimpl TraitTypes may use input lifecycle rather than just types. To control this, you can use a new oneuse<>Boundary syntax, which explicitly declares which generic parameters hidden types can use. Our experience of porting compilers has shown that few changes are needed - most code actually works better under the new default values.

The exception to the above situation is when the function accepts a reference parameter that is only used to read the value and is not included in the return value. An example of this is the following functionindices()It accepts a type of&[T]The slice, but its only function is to read the length, which is used to create an iterator. The slice itself is not required in the return value:

fn indices<'s, T>(
    slice: &'s [T],
) -> impl Iterator<Item = usize> {
    0 .. slice.len()
}

In Rust 2021, this declaration implicitly indicates that the return value is not usedslice. But in Rust 2024, the default situation is exactly the opposite. This means that callers like this will stop compiling in Rust 2024, as they now assumedataBorrowed until iteration completion:

fn main() {
    let mut data = vec![1, 2, 3];
    let i = indices(&data);
    data.push(4);     // <-- !    
    i.next();     // <--  `&data`    
}

This may actually be what you want! This means you can make changes laterindices()The definition, making it really Include in the resultsslice. In other words, the new default values have continuedimpl TraitThe tradition is to preserve the implementation of function changes without compromising the flexibility of the caller.

However, if you no What should I do if I want it? If you want to guaranteeindices()Will not retain its parameters in its return valuesliceWhat about the citation? You can now include ause<>Boundary is used to achieve this, clearly indicating which generic parameters can be included in the return type.

stayindices()In this case, the return type is actually no We would ideally use any generics, so we would writeuse<>：

fn indices<'s, T>(
    slice: &'s [T],
) -> impl Iterator<Item = usize> + use<> {
    //                             -----
        //Return type does not use's' or 'T'`    
    0 .. slice.len()
}

Implement restrictions. Unfortunately, if you really try the above example on Nightly today, you will find that it cannot compile（ Try it yourself [9]）. that is becauseuse<>The boundaries have only been partially implemented: currently, they must always contain at least type parameters. This corresponds to earlier versionsimpl TraitThe restrictions, the latter must Always capture type parameters. In this case, this means that we can write the following content, which can also avoid compilation errors, but still be more conservative than necessary（ Try it yourself [10]）：

fn indices<T>(
    slice: &[T],
) -> impl Iterator<Item = usize> + use<T> {
    0 .. slice.len()
}

This implementation restriction is only temporary and I hope it will be lifted soon! You can Tracking Issue # 130031 [11]Pay attention to the current status.

Alternative solution:'staticBoundary. For not capturing at all whatever Special cases of citation can also be used'staticBoundary, as shown below（ Try it yourself [12]）：

fn indices<'s, T>(
    slice: &'s [T],
) -> impl Iterator<Item = usize> + 'static {
    //                             -------
        //The return type does not capture references.    
    0 .. slice.len()
}

under these circumstances,'staticThe boundary is very convenient, especially considering the current situation surrounding ituse<>The implementation of boundaries is limited, butuse<>The boundaries are generally more flexible, so we hope they will be used more frequently. For example, the compiler has aindicesA variant that returns the index of newtypeIInstead ofusizeValue, therefore it contains ause<I>Declaration.)

conclusion

This example demonstrates how versions can help us remove complexity from Rust. In Rust 2021, when can it be doneimpl TraitThe default rule for using lifecycle parameters in is outdated. They often fail to express user needs and result in the need for obscure solutions. They caused other inconsistencies, such as-> impl FutureAndasync fnReturn position between, or between top-level functions and trait functionsimpl TraitBetween semantics.

Thanks to the version, we were able to solve this problem without breaking the existing code. With the arrival of updated rules in Rust 2024,

Most of the code works fine in Rust 2024, avoiding confusing errors;
For code that requires annotations, we now have a more powerful annotation mechanism that allows you to accurately say what you need to say.

RFC #3617[13]The precise capture was proposed, leaving an unresolved issue regarding syntax, and its tracking problem is#123432[14]。
Unresolved grammar issuesissue #125836[15]The problem was solved by introducing the method used in this article+ use<>Representation method.
Implement restrictions within#130031[16]Tracking in the middle.

reference material

[1]Playground: https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=2448fc4ec9e763c538aaba897433f9b5

[2]Try on Playground:https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=afd9278ac887c0b2fc08bc868200808f

[3]Type alias impl trait:https://rust-lang.github.com/impl-trait-initiative/explainer/tait.html

[4]Try on Playground:https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=b742fbf9b083a6e837db0b170489f34a

[5]be calledCapturesThe obscure solution for trait:https://github.com/rust-lang/rust/issues/34511#issuecomment-373423999

[6]RFC 3425: https://rust-lang.github.io/rfcs/3425-return-position-impl-trait-in-traits.html

[7]To achieve the symmetry we desire:https://hackmd.io/zgairrYRSACgTeZHP1x0Zg

[8]Try it yourself by setting the version in Playground to 2024:https://play.rust-lang.org/?version=nightly&mode=debug&edition=2024&gist=d366396da2fbd5334b7560c3dfb3290b

[9]Try it yourself:https://play.rust-lang.org/?version=nightly&mode=debug&edition=2024&gist=1d6d23ef3813da05ac78a4b97b418f21

[10]Try it yourself:https://play.rust-lang.org/?version=nightly&mode=debug&edition=2024&gist=7965043f4686d5a89b47aa5bfc4f996f

[11]Tracking Issue # 130031:https://github.com/rust-lang/rust/issues/130031