diff --git a/.github/workflows/proposal_validation.yml b/.github/workflows/proposal_validation.yml new file mode 100644 index 00000000..0eb610da --- /dev/null +++ b/.github/workflows/proposal_validation.yml @@ -0,0 +1,54 @@ +name: Proposal Validation + +on: + workflow_call: + inputs: + project: + type: string + description: "Name of project supported by swift-evolution-metadata-extractor tool" + default: swift + +jobs: + run-swift-evolution-metadata-extractor: + name: Validate proposals with swift-evolution-metadata-extractor + runs-on: ubuntu-latest + container: + image: "swift:6.2-noble" + timeout-minutes: 10 + steps: + - name: Checkout swiftlang/swift-evolution-metadata-extractor (main) + uses: actions/checkout@v6 + with: + repository: swiftlang/swift-evolution-metadata-extractor + ref: main + path: seme + + - name: Create cache key + id: cache-key + run: echo "key=seme-$(git -C seme rev-parse HEAD)" >> $GITHUB_OUTPUT + + - name: Restore cache + uses: actions/cache/restore@v5 + id: restore-cache + with: + path: ${{ github.workspace }}/swift-evolution-metadata-extractor + key: ${{ steps.cache-key.outputs.key }} + + - name: Build swift-evolution-metadata-extractor (release) + if: steps.restore-cache.outputs.cache-hit != 'true' + run: | + swift --version + swift build -c release --very-verbose + cp .build/release/swift-evolution-metadata-extractor $GITHUB_WORKSPACE/swift-evolution-metadata-extractor + working-directory: seme + + - name: Save to cache + if: steps.restore-cache.outputs.cache-hit != 'true' + id: save-to-cache + uses: actions/cache/save@v5 + with: + path: ${{ github.workspace }}/swift-evolution-metadata-extractor + key: ${{ steps.cache-key.outputs.key }} + + - name: Validate proposals + run: $GITHUB_WORKSPACE/swift-evolution-metadata-extractor validate --pull-request ${{ github.event.number }} diff --git a/.github/workflows/pull_request.yml b/.github/workflows/pull_request.yml index b451cfa9..a135eb59 100644 --- a/.github/workflows/pull_request.yml +++ b/.github/workflows/pull_request.yml @@ -96,3 +96,10 @@ jobs: with: api_breakage_check_enabled: false license_header_check_project_name: "Swift.org" + + proposal_validation: + name: Test + uses: ./.github/workflows/proposal_validation.yml + with: + with: + project: "swift" diff --git a/README.md b/README.md index 2bb75bc2..de2f40de 100644 --- a/README.md +++ b/README.md @@ -94,6 +94,38 @@ Linked PR: swiftlang/swift-syntax#2859 Enabling cross-PR testing will add about 10s to PR testing time. +### Evolution Proposal Validation + +The proposal validation workflow validates added and changed proposals in a pull request to check for formatting and content errors that will cause metadata extraction to fail or be incomplete. + +To accomplish this, the workflow builds the [swift-evolution-metadata-extractor](https://github.com/swiftlang/swift-evolution-metadata-extractor) tool and runs its `validate` command. To minimize validation times, the built tool is cached and only rebuilt when the tool has changed. + +To use the proposal validation workflow, add a workflow to the repository that contains the directory of proposals. The calling workflow specifies project-specific details such as the directory where the proposals are located. It is only run if a pull request contains changes in the specified directory. + +> [!NOTE] +> The extraction tool currently only supports the evolution proposals of the Swift project at swiftlang/swift-evolution/proposals. The tool and workflow has been designed to be extended to support additional projects in the future. + +An example workflow for Swift Testing which uses a subfolder in the swift-evolution repository: + +```yaml +name: Validate proposals with swift-evolution-metadata-extractor + +on: + pull_request: + types: [opened, reopened, synchronize] + branches: + - 'main' + paths: + - 'proposals/testing/*' + +jobs: + validate: + name: Validate Proposals + uses: swiftlang/github-workflows/.github/workflows/proposal_validation.yml@main + with: + project: "testing" +``` + ## Running workflows locally You can run the Github Actions workflows locally using diff --git a/tests/proposal_validation/0526-deadline copy.md b/tests/proposal_validation/0526-deadline copy.md new file mode 100644 index 00000000..5bcfb358 --- /dev/null +++ b/tests/proposal_validation/0526-deadline copy.md @@ -0,0 +1,713 @@ +# withDeadline + +* Proposal: [SE-0526](0526-deadline.md) +* Authors: [Franz Busch](https://github.com/FranzBusch), [Philippe Hausler](https://github.com/phausler), [Konrad Malawski](https://github.com/ktoso) +* Status: **Returned for revision** +* Implementation: https://github.com/swiftlang/swift/pull/88323 +* Review Manager: [Freddy Kellison-Linn](https://github.com/Jumhyn) +* Review: ([pitch](https://forums.swift.org/t/pitch-withdeadline/85262)) ([review](https://forums.swift.org/t/se-0526-withdeadline/85850)) ([returned for revision](https://forums.swift.org/t/returned-for-revision-se-0526-withdeadline/86438)) ([second review](https://forums.swift.org/t/second-review-se-0526-withdeadline/86791)) ([returned for revision](https://forums.swift.org/t/returned-for-revision-se-0526-withdeadline/87379)) + +## Summary of changes + +This proposal introduces `withDeadline`, a function that executes asynchronous +operations with a composable absolute time limit. The function accepts a +continuous clock instant representing the deadline by which the operation must +complete. If the operation completes before the deadline, the function returns +the result; if the deadline expires first, the operation is cancelled. + +## Motivation + +Asynchronous operations in Swift can run indefinitely, which creates several +problems in real-world applications: + +1. Network operations may not complete when servers become unresponsive, + consuming resources and degrading user experience. +2. Server-side applications need predictable request handling times to maintain + service level agreements and prevent resource exhaustion. +3. Batch processing requires mechanisms to prevent individual tasks from + blocking entire workflows. +4. Resource management becomes difficult when operations lack time bounds, + leading to connection pool exhaustion and memory leaks. +5. Coordinating multiple operations to complete by a shared deadline requires + passing absolute instants, not relative durations that drift through the call + stack. + +Currently, developers must implement timeout logic manually using task groups +and clock sleep operations, resulting in verbose, error-prone code that's +difficult to compose with surrounding async contexts. Each implementation must +carefully handle cancellation, error propagation, and race conditions between +the operation and timer. + +## Proposed solution + +This proposal introduces `withDeadline`, a function that executes an +asynchronous operation with an absolute time limit specified as a clock instant. +This builds upon the clock, instant, and duration types introduced in +[SE-0329](0329-clock-instant-duration.md), the structured concurrency and +cooperative cancellation model from [SE-0304](0304-structured-concurrency.md), +and composes naturally with the task cancellation shields from +[SE-0504](0504-task-cancellation-shields.md). The solution provides a clean, +composable API that handles cancellation and error propagation automatically: + +```swift +let clock = ContinuousClock() + +do { + let result = try await withDeadline(clock.now.advanced(by: .seconds(5)), clock: clock) { + try await fetchDataFromServer() + } + print("Data received: \(result)") +} catch { + print("Request failed: \(error)") +} +``` + +The solution is safer than manual implementations because it handles all race +conditions between the operation and deadline timer, ensures proper cleanup +through structured concurrency, and provides clear semantics for cancellation +behavior. + +## Detailed design + +#### Executing work with a given deadline + +The fundamental entry point for working with deadlines is a single function: `withDeadline`. + +```swift +/// Executes an asynchronous operation with a specified deadline. +/// +/// Use this function to limit the execution time of an asynchronous operation to a specific instant. +/// If the operation completes before the deadline expires, this function returns the result. If the +/// deadline expires first, this function cancels the operation. The `withDeadline` function will +/// return or throw according to how the operation returns or throws as a response to the cancellation. +/// +/// The following example demonstrates using a deadline to limit a network request: +/// +/// ```swift +/// let clock = ContinuousClock() +/// let deadline = clock.now.advanced(by: .seconds(5)) +/// do { +/// let result = try await withDeadline(deadline, clock: clock) { +/// try await fetchDataFromServer() +/// } +/// print("Data received: \(result)") +/// } catch { +/// print("Operation failed") +/// } +/// ``` +/// +/// ## Behavior +/// +/// The function exhibits the following behavior based on deadline and operation completion: +/// +/// - If the operation completes successfully before deadline: Returns the operation result. +/// - If the operation throws an error before deadline: Throws the operation error. +/// - If deadline expires and operation completes successfully: Returns the operation result +/// - If deadline expires and operation throws an error: Throws the operation error. +/// +/// ## Coordinating multiple operations +/// +/// Use `withDeadline` when coordinating multiple operations to complete by the same instant: +/// +/// ```swift +/// let clock = ContinuousClock() +/// let deadline = clock.now.advanced(by: .seconds(10)) +/// +/// async let result1 = withDeadline(deadline, clock: clock) { +/// try await fetchUserData() +/// } +/// async let result2 = withDeadline(deadline) { +/// try await fetchPreferences() +/// } +/// +/// let (user, prefs) = try await (result1, result2) +/// ``` +/// +/// This ensures both operations share the same absolute deadline, avoiding duration drift that can occur +/// when timeouts are passed through multiple call layers. +/// +/// - Important: This function cancels the operation when the deadline expires, but waits for the +/// operation to return. The function may run longer than the time until the deadline if the operation +/// doesn't respond to cancellation immediately. +/// +/// - Parameters: +/// - deadline: The instant by which the operation must complete. +/// - tolerance: The tolerance used for the sleep. +/// - clock: The clock to use for measuring time. +/// - body: The asynchronous operation to complete before the deadline. +/// +/// - Returns: The result of the operation if it completes successfully before or after the deadline expires. +/// +/// - Throws: The error thrown by the operation +nonisolated(nonsending) public func withDeadline( + _ expiration: C.Instant, + tolerance: C.Instant.Duration? = nil, + clock: C = ContinuousClock(), + body: nonisolated(nonsending) () async throws(Failure) -> Return +) async throws(Failure) -> Return +``` + +The deadline-based API accepts a generic `Clock.Instant`, allowing multiple operations +to share the same absolute deadline: + +```swift +let clock = ContinuousClock() +let deadline = clock.now.advanced(by: .seconds(10)) + +async let user = withDeadline(deadline, clock: clock) { + try await fetchUser() +} +async let prefs = withDeadline(deadline, clock: clock) { + try await fetchPreferences() +} + +let (userData, prefsData) = try await (user, prefs) +``` + +These absolute deadlines are composable and nestable to any set scope of a deadline. This means that when more than +one `withDeadline` is nested the minimum of the expiration is taken. If any nested cases are differing clocks the +deadline expires determined by the clock, so no inter-clock conversions need to be computed. This nesting case works +by the outer executing with a given deadline expiration while the inner also executes with its own given deadline +expiration. These two expirations will execute independently to whichever cancels the operation first. Practically +this means that the expiration then is the minimum of the two deadlines, without needing to compare or calculate +between them. + + +```swift +let clock = ContinuousClock() +let userAndPrefsDeadline = clock.now.advanced(by: .seconds(5)) + +let userAndPrefs = try await withDeadline(userAndPrefsDeadline, clock: clock) { + let user = try await fetchUser() + let prefs = try await fetchPrefs() +} + +func fetchPrefs() async throws(FetchFailure) -> Prefs { + let prefsDeadline = clock.now.advanced(by: .seconds(10)) + do { + return try await withDeadline(prefsDeadline, clock: clock) { + try await fetchPreferences() + } + } catch { + throw error.underlyingError + } +} +``` + +Particularly in this case the composition can be made such that two independent regions can participate in a composed +deadline across library boundaries and still result in the correct deadline for the composed expectation of the caller. +This is achieved due to the fact that each nesting of `withDeadline` will independently apply a deadline expiration. +The first to cancel will be the composition of the effective minimum no matter the clock specified. This means that +there is no need for a current deadline for the service of calculating which is the minimum execution deadline. + +#### Shorthand for quickly using common deadline construction + +Constructing an instant every time is not per se the most terse; so a simple extension offers the ease of construction +with the same compositional advantage as the primary entry point. + +```swift +nonisolated(nonsending) public func withDeadline( + in timeout: C.Instant.Duration, + tolerance: C.Instant.Duration? = nil, + clock: C = ContinuousClock(), + body: nonisolated(nonsending) () async throws(Failure) -> Return +) async throws(Failure) -> Return + +nonisolated(nonsending) public func withDeadline( + in timeout: ContinuousClock.Instant.Duration, + tolerance: ContinuousClock.Instant.Duration? = nil, + body: nonisolated(nonsending) () async throws(Failure) -> Return +) async throws(Failure) -> Return +``` + +The implementation of this is trivially: + +```swift +try await withDeadline(clock.now.advanced(by: timeout), tolerance: tolerance, body: body) +``` + +#### Non-escaping nonisolated(nonsending) operation closure + +Many existing deadline/timeout implementations require a `@Sendable` and +`@escaping` closure which makes it hard to compose in isolated context and use +non-Sendable types. This design ensures that the closure is both non-escaping +and nonisolated(nonsending) for composability: + +```swift +actor DataProcessor { + var cache: [String: Data] = [:] + + func fetchWithDeadline(url: String) async throws { + // The closure can access actor-isolated state because it's nonisolated(nonsending) + let data = try await withDeadline(in: .seconds(5)) { + if let cached = cache[url] { + return cached + } + return try await URLSession.shared.data(from: URL(string: url)!) + } + cache[url] = data + } +} +``` + +If the closure were `@Sendable`, it couldn't access actor-isolated state like +`cache`. The `nonisolated(nonsending)` annotation allows the closure to compose +with surrounding code regardless of isolation context, while maintaining safety +guarantees. + +#### Cancellation + +This API uses the base cancellation to communicate the expiration of the deadline. +The information to differentiate a cancellation due to normal task cancellation is +expanded to handle two new forms of cancellation; a cancellation due to deadline expiration, +and a custom cancellation with a specified string for a reason. Since this is not a closed +set of possible reasons for future development, this reason is left as an open enumeration. + +Today `CancellationError` is an empty type with no payload or information conveyed to indicate +the reasoning for cancellation. [SE-0304](0304-structured-concurrency.md) originally noted that +"no information is passed to the task about why it was cancelled," treating cancellation as a +lightweight, uniform signal. With the introduction of deadlines, however, differentiating between +a cancellation due to deadline expiration and a cancellation from an explicit `Task.cancel()` call +becomes practically necessary for correct error reporting and recovery. A new sub-type will be +added to represent the reason for the cancellation, a new initializer for `CancellationError` will +be added for constructing a `CancellationError` with a given reason, and a new property will be +added for determining what the reason of the cancellation was. This modification not only allows +for developers to express the difference between a cancellation due to deadline expiration versus +normal task cancellation, but also express a custom reason for indicating why something might be +cancelled. + +```swift +public struct CancellationError: Error { + @nonexhaustive + public enum Reason { + case taskCancelled + case deadlineExpired + case custom(String) + } + + public var reason: Reason { get } + public init(reason: Reason) + + // This is shorthand for `CancellationError(reason: .taskCancelled)` + public init() +} +``` + +Switching upon the reason specifically will require the developer to handle unknown cases since +there may be situations in which additional cases may be added at a future point. Because +`CancellationError.Reason` is defined in the Concurrency module (which ships as part of the +standard library and is a resilient module), the enum is non-frozen by default and switch +statements require an `@unknown default` case. Since previous cancellation was something that +has been already written the developer already has handled the cases of cancellation without a +given reason; this will continue to be the case. + +To aid in the population of cancellation errors, new APIs will be added. These will all be cases +where a task or child task is cancelled and a CancellationError would normally be created. + +```swift +extension Task { + public func cancel(reason: CancellationError.Reason) +} + +extension UnsafeCurrentTask { + public func cancel(reason: CancellationError.Reason) +} + +extension TaskGroup { + public func cancelAll(reason: CancellationError.Reason) +} + +extension ThrowingTaskGroup { + public func cancelAll(reason: CancellationError.Reason) +} + +extension DiscardingTaskGroup { + public func cancelAll(reason: CancellationError.Reason) +} + +extension ThrowingDiscardingTaskGroup { + public func cancelAll(reason: CancellationError.Reason) +} +``` + +#### Failures and expiration + +The withDeadline throwing behavior is that of the operation's throwing behavior. If the operation throws a +specific type then the withDeadline will throw that same type, this permits the case where a cancellation aware +throwing behavior is then respected with the most information possible and specifically does not throw away +the potential failure information. This means that if a developer wishes to communicate a failure solely due to +deadline expiration, the cancellation error that is thrown should then contain the reason of `.deadlineExpired`. + +This error is propagated from whenever the task (or child task) is cancelled via the `cancel(reason:)` method. +The reason specified will then be available to the `CancellationError` and can be retrieved from the `reason` +property on the cancellation error. + +### Behavioral Details + +1. The user specified closure runs concurrently to the timing of the expiration +of the deadline. +2. The first event between the closure and the timing determines the result. +3. When either the expiration happens such that the deadline is hit or the user +specified closure is done, the unfinished part of the execution is cancelled. +4. If the deadline expires first, the operation is cancelled but the function waits + for it to return. +5. The function handles both the operation's result and any errors thrown. + +The function cancels the operation when the deadline expires, but waits for the +operation to return. This means `withDeadline` may run longer than the time until +the deadline if the operation doesn't respond to cancellation immediately. This +design ensures proper cleanup and prevents resource leaks from abandoned tasks. + +Users who wish to adjust behaviors can use the task cancellation shields and/or +task cancellation handlers to alter the behavior of the return values. These in +conjunction with manual processing of do/catch clauses can compose to complex +behaviors needed for many specialized scenarios. + +#### Behaviors for Cancellation and Expiration + +The following examples outline common composition and cancellation behaviors. + +- **Example 0**: Operation completes before deadline - returns successfully with no error. +- **Example 1**: Operation throws before deadline - the thrown error propagates. +- **Example 2**: Inner deadline (2s) expires before outer deadline (3s) - only the inner cancellation handler fires, and the operation's thrown error propagates. +- **Example 3**: Outer deadline (2s) expires before inner deadline (3s) - both cancellation handlers fire because cancellation propagates inward through the task tree. +- **Example 4**: Outer deadline (2s) expires before inner deadline (10s), but the sleep is shorter (3s) - both handlers still fire at the 2s mark because the outer deadline governs. +- **Example 5**: Demonstrates that `withDeadline` waits for the operation to return even after cancellation. The busy-loop ignores cancellation and runs for the full 10 seconds despite the 2s inner deadline. + +```swift +struct LocalError: Error { } + +print("====== EXAMPLE 0 ======") +do { + let value = try await withDeadline(in: .seconds(2), tolerance: .microseconds(2)) { + return "Success" + } +} catch { + print("caught \(error)") +} +// ====== EXAMPLE 0 ====== + +print("====== EXAMPLE 1 ======") +do { + try await withDeadline(in: .seconds(2), tolerance: .microseconds(2)) { + throw LocalError() + } +} catch { + print("caught \(error)") +} +// ====== EXAMPLE 1 ====== +// caught LocalError() + +print("====== EXAMPLE 2 ======") +do { + try await withDeadline(in: .seconds(3), tolerance: .microseconds(2)) { + try await withTaskCancellationHandler { + try await withDeadline(in: .seconds(2), tolerance: .microseconds(2)) { + try await withTaskCancellationHandler { + let elapsed = await ContinuousClock().measure { + try? await Task.sleep(for: .seconds(10)) + } + print("\(elapsed) elapsed") + throw LocalError() + } onCancel: { + print("cancel inner") + } + } + } onCancel: { + print("cancel outer") + } + } +} catch { + print("caught \(error)") +} +// ====== EXAMPLE 2 ====== +// cancel inner +// 2.001315 seconds elapsed +// caught LocalError() + +print("====== EXAMPLE 3 ======") +do { + try await withDeadline(in: .seconds(2), tolerance: .microseconds(2)) { + try await withTaskCancellationHandler { + try await withDeadline(in: .seconds(3), tolerance: .microseconds(2)) { + try await withTaskCancellationHandler { + let elapsed = await ContinuousClock().measure { + try? await Task.sleep(for: .seconds(10)) + } + print("\(elapsed) elapsed") + throw LocalError() + } onCancel: { + print("cancel inner") + } + } + } onCancel: { + print("cancel outer") + } + } +} catch { + print("caught \(error)") +} +// ====== EXAMPLE 3 ====== +// cancel inner +// cancel outer +// 2.00507375 seconds elapsed +// caught LocalError() + +print("====== EXAMPLE 4 ======") +do { + try await withDeadline(in: .seconds(2), tolerance: .microseconds(2)) { + try await withTaskCancellationHandler { + try await withDeadline(in: .seconds(10), tolerance: .microseconds(2)) { + try await withTaskCancellationHandler { + let elapsed = await ContinuousClock().measure { + try? await Task.sleep(for: .seconds(3)) + } + print("\(elapsed) elapsed") + throw LocalError() + } onCancel: { + print("cancel inner") + } + } + } onCancel: { + print("cancel outer") + } + } +} catch { + print("caught \(error)") +} +// ====== EXAMPLE 4 ====== +// cancel inner +// cancel outer +// 2.005246625 seconds elapsed +// caught LocalError() + +print("====== EXAMPLE 5 ======") +do { + try await withDeadline(in: .seconds(3), tolerance: .microseconds(2)) { + try await withTaskCancellationHandler { + try await withDeadline(in: .seconds(2), tolerance: .microseconds(2)) { + try await withTaskCancellationHandler { + let clock = ContinuousClock() + let elapsed = await clock.measure { + let start = clock.now + while clock.now < start + .seconds(10) { + await Task.yield() + } + } + print("\(elapsed) elapsed") + throw LocalError() + } onCancel: { + print("cancel inner") + } + } + } onCancel: { + print("cancel outer") + } + } +} catch { + print("caught \(error)") +} +// ====== EXAMPLE 5 ====== +// cancel inner +// cancel outer +// 10.000002291000001 seconds elapsed +// caught LocalError() +``` + +## Source compatibility + +The proposed APIs are additive and the behavior of deadlines are composed +without a need for intermediary participation. Existing systems that handle +cancellation or throwing of errors will compose with this without the need +to adjust for the new deadline semantics. + +## Effect on ABI compatibility + +Since this is an additive proposal there is no change to any existing ABI. +The modification to `CancellationError` adds a new stored property and initializer +but preserves the existing default initializer with identical behavior - existing +code that constructs `CancellationError()` will continue to produce an error with +the equivalent of `.taskCancelled` as its reason. The proposed APIs are capable of +being implemented in less performant manners prior to the introduction of typed throws. +Back porting this feature is not a proposed part of the pitch but no technical +limitation is added except the burden of making the implementation fragmented upon +deployment. + +## Effect on API resilience + +This is an additive API and no existing systems are changed, however it will +introduce a few new types that will need to be maintained as ABI interfaces. + +## Location and availability + +Previously this feature was pitched for swift-async-algorithms. However due +to the large demand and existing requests for this feature it was considered +perhaps not specialized enough to live in that particular package. It would +be a fairly common occurrence to need this functionality and it would be better +served living in the Concurrency module. + +The availability has particular consideration listed in the ABI section. + +## Future directions + +This can have an impact upon executors. The current implementation does not +need executors to do anything different than they do as this is pitched, but +some modification around cancellation of jobs could be added to allow +executors to more efficiently handle deadlines. + +There are potentials of exposing the underlying structured concurrency primitives +to enable APIs like `withDeadline`. This has a precedent in other languages +and is often called `race`. Introducing the `withDeadline` API does not preclude +that as an eventuality and lends credence to the utility of having that as +a general purpose feature, however offering the more primitive functionality +would still likely have the same considerations and motivation for introducing +a `withDeadline` API. So these concepts are not mutually exclusive or blocking. +If at some point in time the concurrency library grows a new `race` type +primitive, then `withDeadline` would likely be a strong candidate for using that. + +## Alternatives considered + +### Timeout-based API instead of Deadline-based API + +An earlier design considered naming the primary API `withTimeout` and having it +accept a duration parameter instead of focusing on deadline-based +(instant-based) semantics: + +```swift +public func withTimeout( + in duration: Duration, + body: () async throws(Failure) -> Return +) async throws(TimeoutError) -> Return +``` + +This approach was rejected because deadline-based APIs provide better +composability and semantics. Duration-based timeouts accumulate drift when +passed through multiple call layers, making it impossible to guarantee that +nested operations complete within a precise time window, whereas absolute +deadlines allow multiple operations to coordinate on the same completion +instant. Consider a function that applies a 10-second timeout and then calls +two sub-operations each with the remaining time: the overhead of each call +layer (scheduling, argument evaluation, function prologues) silently erodes +the budget, and the second sub-operation receives a shorter effective timeout +than intended. With an absolute deadline, every layer in the stack sees the +same instant and no time is lost in translation. + +This is the same reasoning behind Go's `context.WithDeadline` - Go provides +both `WithTimeout` (relative) and `WithDeadline` (absolute), but recommends +deadlines for composable, multi-layer operations because the absolute instant +propagates without drift. Kotlin's `withTimeout` is duration-based, but +Kotlin's coroutine scope carries a single deadline internally and computes the +minimum against any new timeout, which is effectively what the nested +`withDeadline` composition in this proposal achieves explicitly. + +The rejection however does not apply when the funnel point of the deadline +functionality is sent to an entry point handling the composition by using +instants and composing with the current deadline by a minimum function. + +### @Sendable and @escaping Closure + +An earlier design considered using `@Sendable` and `@escaping` for the closure +parameter. This approach was rejected because it severely limited composability. The +`@Sendable` requirement prevented accessing actor-isolated state, making it +difficult to use in isolated contexts. The final design uses +`nonisolated(nonsending)` to enable better composition while maintaining safety. + +### Naming + +The naming of this API has a notable lineage: during the development of +[SE-0329](0329-clock-instant-duration.md), the type now called `Instant` was +originally named `Deadline` (v1.1), and was later renamed to `Instant` because +that name better describes a general-purpose point in time. The name `Deadline` +is now reclaimed for its original intended purpose - expressing a temporal bound +by which work must complete - while `Instant` serves as the underlying type that +represents the point in time. + +Some feedback was posed to name this function around the cancellation behavior; +along the lines of `withAutomaticTaskCancellation`. This naming does not focus +upon the time related qualities of the concept of deadlines, which is the primary +behavioral aspect of this API. The cancellation behavior is part of the realities +of how the language level concept of cooperative cancellation works and in reading +the code at a potential call site it is more meaningful to convey the temporal +nature of a deadline than to convey the cancellation being automatic. Immediately +the question that would be posed by folks unaware of this new API would be: +"What automatic mechanism makes that cancellation happen?" rather than realizing +without ambiguity that a concept of time is involved by knowing it is a deadline. + +Since the closure may itself use `withTaskCancellationHandler` or catch cancellation +errors to return a nullable result or some other partial result it then makes the most +sense to even avoid names like `withCancellationDeadline`. + +One proposed name that does make some sense to infer the cooperative cancellation nature +of `withDeadline` was a name of `withTaskDeadline` to infer the interoperation with +the Concurrency primitive Task (and TaskGroup's child tasks). Even though that naming +wise this has more appeal than other alternative names the major issue is that +there is no real potential of any other deadline being introduced. So the `Task` portion +of that name is extraneous. + +From a nomenclature standpoint, `withDeadline` would be a term of art for Swift. By its +nature has an implication of cooperative cancellation due to the design of Swift's +concurrency runtime and by that implication also interacts solely with tasks. This +follows suit with other languages like Kotlin - the naming in that case is withTimeout +because the timeout in that case is an elapsed duration instead of a deadline instant. +The name `withDeadline` also reads naturally at the call site - `try await withDeadline(...)` +immediately communicates to the reader that a temporal bound is in effect, which aids code +review and debugging. Names centered on the mechanism (`withAutomaticTaskCancellation`) +require the reader to infer the temporal aspect, while names centered on the concept +(`withDeadline`) let the reader infer the mechanism from context. + +### Previous Incarnations + +The clock was originally suggested as a generic clock originally, however when +moving to a composable interface the clock was made to be concrete to the +`ContinuousClock`. This ended up being too restrictive so that was relaxed to +where a generic clock was used but restricted to a clock with the `Instant.Duration` +that is `Swift.Duration`. This constraint allows for the composition of expirations +and in the cases of differing clocks an approximation of the expiry is made by +using the delta from now as an offset. + +### Separate DeadlineExceededError type + +An alternative design would introduce a distinct `DeadlineExceededError` type rather +than extending `CancellationError` with a `Reason`. This was considered and rejected +for several reasons: + +1. **Typed throws compatibility**: Because `withDeadline` preserves the typed failure + of the operation closure via `throws(Failure)`, introducing a new error type would + require a wrapper like `TimeoutError` that conflates two concerns - the + deadline expiration and the operation's own error domain. This forces every caller + to destructure a wrapper type even in the common case where they simply want to + know whether the operation failed. +2. **Composability with existing cancellation handlers**: Code that already uses + `withTaskCancellationHandler` or checks `Task.isCancelled` would not observe a + `DeadlineExceededError` - it would appear as an ordinary error rather than a + cancellation. By expressing deadline expiration as a reason on `CancellationError`, + all existing cancellation-aware code automatically participates in deadline behavior. +3. **Consistency with the cooperative cancellation model**: Deadline expiration is + mechanically a cancellation - the task is cancelled and the operation responds + cooperatively. Using the same error type with an enriched reason preserves this + semantic identity rather than introducing a parallel concept. + +### Task-installed deadlines + +[SE-0304](0304-structured-concurrency.md) originally envisioned that "a deadline can +be installed on a task and naturally propagate through arbitrary levels of API, including +to child tasks." An alternative design following this model would attach a deadline +directly to the task, making it implicitly visible to all child tasks without explicit +nesting. This approach was not taken because: + +1. Implicit propagation through task-local state would make it difficult to reason about + which deadline is in effect at any given point, especially when library code installs + its own deadlines. +2. The explicit nesting model composes transparently - each `withDeadline` scope is + visible in the source code, and the minimum-expiration composition rule is easy to + reason about. +3. Nothing in this proposal precludes a future task-installed deadline mechanism; the + explicit `withDeadline` API would remain useful even if such a mechanism were added. + +## Changelog +- 1.1 Returned for revision + - The typed throws signature was altered to avoid an extra error type + - Removed the restriction around the instant requiring the duration type to be `Swift.Duration` + - The accessor for the current deadline was removed due to difficulty for using any InstantProtocol + - A new interface on CancellationError was added to handle the reasons for why a task or child task is cancelled (including a deadline exceeded reason). + - 1.0 Initial revision diff --git a/tests/proposal_validation/0527-rigidarray-uniquearray copy.md b/tests/proposal_validation/0527-rigidarray-uniquearray copy.md new file mode 100644 index 00000000..12facb6f --- /dev/null +++ b/tests/proposal_validation/0527-rigidarray-uniquearray copy.md @@ -0,0 +1,1871 @@ +# UniqueArray + +* Proposal: [SE-0527](0527-rigidarray-uniquearray.md) +* Authors: [Karoy Lorentey](https://github.com/lorentey), [Alejandro Alonso](https://github.com/Azoy) +* Review Manager: [Steve Canon](https://github.com/stephentyrone) +* Status: **Accepted** +* Implementation: [swiftlang/swift#87521](https://github.com/swiftlang/swift/pull/87521) +* Review: ([pitch](https://forums.swift.org/t/pitch-rigidarray-and-uniquearray/85455)) + ([first review](https://forums.swift.org/t/se-0527-rigidarray-and-uniquearray/85985)) + ([second review](https://forums.swift.org/t/focused-re-review-se-0527-uniquearray-reallocate-capacity/86944)) + ([acceptance](https://forums.swift.org/t/accepted-with-modifications-se-0527-uniquearray/87282)) + +[swift-collections]: https://github.com/apple/swift-collections + +## Summary of changes + +We propose to introduce two new array types to the Swift Standard Library, +`RigidArray` and `UniqueArray`, that are both capable of storing noncopyable +elements. + +## Motivation + +Swift 5.9 introduced noncopyable struct and enum types into the language and we've +steadily been adding new API that helps support such types. The Standard +Library has implemented noncopyable types of its own, like `Atomic` and `Mutex`, +as well as the `InlineArray` container type that allows storing values of +potentially noncopyable types in inline storage. However, the Standard Library +is still lacking resizable data structure implementations that +support noncopyable elements. + +As Swift developers, we reach for `Array` when we need to store a dynamically +resizable list of values with efficient operations for accessing them. +Unfortunately, the classic `Array` does not support noncopyable values. There +are two ways we could try to shoehorn support for noncopyable elements onto it: + +1. One idea is to keep `Array` copyable, and tweak its mutation operations to + ensure uniqueness in some way other than copying noncopyable elements. + For example, we could have mutations trigger a runtime error, or we could add + new arguments to mutations that describe specifically how to clone elements. + In practice, neither of these options lead to an acceptable programming + experience. + +2. A (superficially) more attractive idea would be to make `Array` + _conditionally copyable_, depending on the copyability of its elements. + Mutation operations would then need to gain an additional runtime condition + that dispatches to the copy-on-write path if and only if `Element` happens + to be copyable, otherwise assuming uniqueness. There are two technical + issues here: + + 1. Swift code is currently unable to check whether a type argument is + copyable at runtime, and conditionally copy instances if so. + 2. The need for such a "conformance" check would add potential overhead to + every array mutation. + + The first problem is resolvable, but the second would be a difficult one to + swallow -- especially as it would particularly impact generic + contexts that allow `Element` to be noncopyable. The check for + copyability needs to be a runtime condition in such contexts: we cannot have + `append` assume unique storage just because it is invoked in a context that + _allows_ noncopyable elements. Even if we decide to spend resources on + improving the optimizer in this area, it wouldn't be possible to optimize the + condition away in every case, and consulting the Swift runtime every time + a function needs to mutate an array instance seems unlikely to be acceptable. + +Obviously, making `Array`'s performance even more tricky to analyze than it +already is would work directly against the goals of the +[Swift Ownership Manifesto][ownership-manifesto], which led to the introduction +of noncopyable types in the first place. Our goal is not just to have an array +of noncopyables -- we need to do it with predictably good performance that is +easy to analyze. + +[ownership-manifesto]: https://github.com/swiftlang/swift/blob/main/docs/OwnershipManifesto.md + +Beyond its inherent lack of support for noncopyable elements, `Array` has two +major sources of unpredictable complexity spikes that make it unpalatable to +performance-minded use cases: + +1. `Array` has **copy-on-write value semantics**, and it's relatively easy to + mutate a shared copy by accident. Every time we do that, the operation needs + to allocate a full copy of the entire array, turning even "usually" + constant-complexity operations like a simple subscript reassignment into + linear-complexity monsters. Use cases that cannot accept such irregular + performance spikes need to carefully avoid making copies, and + there is no indication if/when they get it wrong. + +2. `Array` is a **dynamic data structure**: it implicitly resizes itself as needed + to accommodate the items added to it. This resizing is done by allocating a + brand new buffer of the appropriate size, and copying or moving all existing + elements into it. This happens automatically, and it leaves no mark in the + source: the "same" `append` invocation will run in constant space and time + in most cases, but once in a blue moon it triggers resizing and it suddenly + becomes linear. The geometric growth pattern ensures that `append` will still + average out into "amortized" O(1) complexity, but its _actual_ worse-case + complexity is O(`count`). Use cases that cannot accept such irregular + performance spikes need to go out of their way to reserve enough capacity in + advance, and getting it wrong leads to no obvious error. + +These two features aren't inherently wrong -- in fact, they both have highly +desirable benefits, as they greatly simplify Swift's programming model. When +using `Array`, its copy-on-write value semantics means that copies can be +cheaply made, and functions are empowered to hold onto array instances whenever +they want, without having to change their interface, or even letting the caller +know about it. Similarly, dynamic sizing lets us avoid having to constantly +think about how much memory an operation will need to do its job. + +However, when we use Swift in contexts where we need to ensure +reliably high performance, then these features tend to get in the way of +achieving that, by making it significantly more difficult to analyze or +guarantee how the code will behave at runtime. + +Complicating `Array` by bolting even more features onto it is not going to let +us succeed here; what we actually need are _additional_ array implementations +that are optimized specifically for use cases that require more predictable +performance than what `Array` can provide. (Having several implementations for +the same underlying data structure is not a radical idea; indeed, the Standard +Library's own `ContiguousArray` and `Foundation`'s `NSArray` are preexisting +resizable array types, doing away with `Array` features that are undesirable in +some contexts: Objective-C bridging and value semantics, respectively.) + +But how many new array types do we need? The two features above are technically +orthogonal to each other, and we could independently turn them on or off as +needed. This suggests four hypothetical array variants, with one of them being +the existing `Array` type: + +| | **Noncopyable** | **Copy-on-write** | +| ---: | :---: | :---: | +| **Fixed capacity** | ??? | ??? | +| **Dynamic** | ??? | `Array` | + +The primary reason to reach for a fixed-capacity data structure is to avoid +implicit allocations, but copy-on-write behavior would be in direct conflict +with that. This means we can leave the top right corner empty, leaving us with +this table. + +| | **Noncopyable** | **Copy-on-write** | +| ---: | :---: | :---: | +| **Fixed capacity** | ??? | --- | +| **Dynamic** | ??? | `Array` | + +## Proposed solution + +We propose to add two new array types to the Swift Standard Library: +`RigidArray` and `UniqueArray`. + +Both of these are true array types, providing familiar array operations: we can +append, insert, replace, remove elements, reorder them in arbitrary ways, and +quickly access their contents using integer offsets as indices. They both use +a single, heap-allocated, contiguous memory region as storage, allowing it to +be partially initialized, with initialized items collected at the front -- in a +nutshell, they implement the classic variable-sized array data structure, just +like the preexisting `Array` and `ContiguousArray` types. + +These types come included in a new module in the Swift toolchain named +`Containers`. Like `Collections` from [swift-collections], this module will be +a home for future data structure implementations like ring buffers. More +examples of potential data structures are included in +[future directions](#Rigid-and-unique-variants-of-other-standard-data-structures). + +### `UniqueArray` + +`UniqueArray` is a great choice for general high-performance contexts where we +want to avoid using copy-on-write containers, but we aren't overly concerned +about strictly budgeting memory, and we just want a simple, dynamically +resizing array type, along the lines of `std::vector` in C++, or `Vec` in Rust. + +`UniqueArray` gives us an `Array` variant whose storage is always +_uniquely held_. This is statically enforced, by declaring `UniqueArray` as a +noncopyable type: a `UniqueArray` itself can only ever be held by +a single variable (stored property, local variable, function argument, +etc) at any one time, and it can only be mutated through that single variable. +It is possible to move the array to another variable, but this consumes the +original, rendering it unusable/uninitialized. For example, the `var b = a` +statement in the example below is a move operation, not a copy: + +```swift +import Containers + +struct FileHandle: ~Copyable { + let fd: UInt32 + + init(reading path: String) throws { fd = try open(path, .read) } + + deinit { + try! close(fd) + } +} + +let foo = try FileHandle(reading: "foo.txt") +let bar = try FileHandle(reading: "bar.md") + +var a = UniqueArray() +a.append(foo) // OK, consumes `foo` +a.append(bar) // OK, consumes `bar` + +var b = a // OK, consumes `a`, moving the array instance into `b` + +b.append(try FileHandle(reading: "baz.swift")) // OK +// `b` now contains open handles for foo.txt, bar.md, and baz.swift + +a.append(try FileHandle(reading: "Info.plist")) // error: `a` used after consume (used here) +``` + +By virtue of being noncopyable itself, `UniqueArray` is naturally able to hold +noncopyable elements like the (strictly illustrative) file handles in the +example above. Array operations that take elements have been carefully designed +to take ownership into account, and they have been annotated with +`consuming` or `borrowing` keywords to explain how they interact with element +ownership. + +As expected of any proper dynamically resizing container, `UniqueArray` relies +on a geometric growth curve to ensure acceptable (amortized) performance: when +it needs to resize itself, it does so by multiplying its previous capacity by +some constant factor, rather than simply growing itself linearly to cover the +operation at hand. The growth factor is an internal implementation detail and +it is subject to change between environments, platforms and Swift releases; it +is not user-configurable. + +### `RigidArray` + +For the lowest-level use cases (such as core systems programming, +memory-constrained embedded platforms, or realtime contexts), `UniqueArray` is +not quite enough: we also have a clear need for a fixed-capacity noncopyable +array type. + +For example, when we are trying to write Swift code for an environment where +available memory is measured in _kilobytes_, we want every allocation to be +explicit in the source, so that we are forced to precisely account and budget +for it. In these contexts, there is no room for container types that helpfully +resize or copy their storage whenever they feel like it -- we are quite happy +to give up that flexibility in exchange for careful, pedantic control. +`RigidArray` is intended to cater to such use cases; its name reflects its +inflexible, _rigid_ nature. + +`RigidArray` instances are always allocated with a specific capacity, and they +must operate entirely within that. Consequently, they can become full, when +they are no longer able to accommodate any new items. Attempting to add a new +value to a full `RigidArray` results in a runtime trap: + +```swift +var c = RigidArray(capacity: 2) +print(c.isFull) // => false +print(c.freeCapacity) // => 2 + +c.append(23) +print(c.isFull) // => false +print(c.freeCapacity) // => 1 + +c.append(42) +print(c.isFull) // => true +print(c.freeCapacity) // => 0 + +c.append(7) // runtime error: RigidArray capacity overflow +``` + +Treating this as a precondition violation rather than a recoverable error allows +`RigidArray` to provide the same basic operations as `UniqueArray`. This +preserves a path towards unifying them under [an ownership-aware +`RangeReplaceableCollection`-like abstraction][RangeReplaceableContainer]. It +also avoids the need to over-complicate `RigidArray`'s operations by forcing +them to report failure in some recoverable way. + +[RangeReplaceableContainer]: https://github.com/apple/swift-collections/blob/1.4.1/Sources/ContainersPreview/Protocols/Container/RangeReplaceableContainer.swift + +In practice, overflowing `RigidArray` storage indeed feels like a programmer +error: it indicates a misuse of the type, rather than a routine issue. Trying to +remove the last item from an empty `Array` results in a trap -- and so trying +to append one to a full `RigidArray` also naturally results in one. + +While `RigidArray` never resizes itself automatically, its capacity is not +actually part of its type: rigid array instances are in fact arbitrarily +resizable using a `reallocate` operation that can be explicitly invoked +to grow (or shrink) the array's storage: + +```swift +var d = RigidArray(capacity: 2) +d.append(10) +d.append(20) +print(d.isFull) // => true +print(d.freeCapacity) // => 0 + +d.reallocate(capacity: 10) +print(d.isFull) // => false +print(d.freeCapacity) // => 8 + +d.append(30) // OK! +``` + +The array allocates precisely as much storage as requested -- neither more nor +less. This operation lets us use `RigidArray` to implement wrapper types that +implement arrays with arbitrary, custom resizing logic. In fact, `UniqueArray` +is itself implemented as such. + + +## Detailed design + +### `RigidArray` + +```swift +/// A fixed capacity, heap allocated, noncopyable array of potentially +/// noncopyable elements. +/// +/// `RigidArray` instances are created with a specific maximum capacity. Elements +/// can be added to the array up to that capacity, but no more: trying to add an +/// item to a full array results in a runtime trap. +/// +/// var items = RigidArray(capacity: 2) +/// items.append(1) +/// items.append(2) +/// items.append(3) // Runtime error: RigidArray capacity overflow +/// +/// Rigid arrays provide convenience properties to help verify that they have +/// enough available capacity: `isFull` and `freeCapacity`. +/// +/// guard items.freeCapacity >= 4 else { throw CapacityOverflow() } +/// items.append(copying: newItems) +/// +/// It is possible to extend or shrink the capacity of a rigid array instance, +/// but this needs to be done explicitly, with operations dedicated to this +/// purpose (such as ``reserveCapacity`` and ``reallocate(capacity:)``). +/// The array never resizes itself automatically. +/// +/// It therefore requires careful manual analysis or up front runtime capacity +/// checks to prevent the array from overflowing its storage. This makes +/// this type more difficult to use than a dynamic array. However, it allows +/// this construct to provide predictably stable performance. +/// +/// This trading of usability in favor of stable performance limits `RigidArray` +/// to the most resource-constrained of use cases, such as space-constrained +/// environments that require carefully accounting of every heap allocation, or +/// time-constrained applications that cannot accommodate unexpected latency +/// spikes due to a reallocation getting triggered at an inopportune moment. +/// +/// For use cases outside of these narrow domains, we generally recommend +/// the use of ``UniqueArray`` rather than `RigidArray`. (For copyable elements, +/// the standard `Array` is an even more convenient choice.) +@frozen +public struct RigidArray: ~Copyable {} + +extension RigidArray: Sendable where Element: Sendable & ~Copyable {} +``` + +### `UniqueArray` + +```swift +/// A dynamically self-resizing, heap allocated, noncopyable array of +/// potentially noncopyable elements. +/// +/// `UniqueArray` instances automatically resize their underlying storage as +/// needed to accommodate newly inserted items, using a geometric growth curve. +/// This frees code using `UniqueArray` from having to allocate enough +/// capacity in advance; on the other hand, it makes it difficult to tell +/// when and where such reallocations may happen. +/// +/// For example, appending an element to a dynamic array has highly variable +/// complexity; often, it runs at a constant cost, but if the operation has to +/// resize storage, then the cost of an individual append suddenly becomes +/// proportional to the size of the whole array. +/// +/// The geometric growth curve allows the cost of such latency spikes to +/// get amortized across repeated invocations, bringing the average cost back +/// to O(1); but they make this construct less suitable for use cases that +/// expect predictable, consistent performance on every operation. +/// +/// Implicit growth also makes it more difficult to predict/analyze the amount +/// of memory an algorithm would need. Developers targeting environments with +/// stringent limits on heap allocations may prefer to avoid using dynamically +/// resizing array types as a matter of policy. The type `RigidArray` provides +/// a fixed-capacity array variant that caters specifically for these use cases, +/// trading ease-of-use for more consistent/predictable execution. +@frozen +public struct UniqueArray: ~Copyable {} + +extension UniqueArray: Sendable where Element: Sendable & ~Copyable {} +``` + +### API on _both_ `RigidArray` and `UniqueArray` + +#### Basics + +```swift +extension [Rigid|Unique]Array where Element: ~Copyable { + /// The maximum number of elements this array can hold without having to + /// reallocate its storage. + /// + /// - Complexity: O(1) + public var capacity: Int { + get + } + + /// The number of additional elements that can be added to this array without + /// reallocating its storage. + /// + /// - Complexity: O(1) + public var freeCapacity: Int { + get + } + + /// A span over the elements of this array, providing direct read-only access. + /// + /// - Complexity: O(1) + public var span: Span { + get + } + + /// A mutable span over the elements of this array, providing direct + /// mutating access. + /// + /// - Complexity: O(1) + public var mutableSpan: MutableSpan { + mutating get + } + + public func isTriviallyIdentical(to: borrowing Self) -> Bool + + /// Arbitrarily edit the storage underlying this array by invoking a + /// user-supplied closure with a mutable `OutputSpan` view over it. + /// This method calls its function argument at most once, allowing it to + /// arbitrarily modify the contents of the output span it is given. + /// The argument is free to add, remove or reorder any items; however, + /// it is not allowed to replace the span or change its capacity. + /// + /// When the function argument finishes (whether by returning or throwing an + /// error) the {rigid|unique} array instance is updated to match the final contents of + /// the output span. + /// + /// - Parameter body: A function that edits the contents of this array through + /// an `OutputSpan` argument. This method invokes this function + /// at most once. + /// - Returns: This method returns the result of its function argument. + /// - Complexity: Adds O(1) overhead to the complexity of the function + /// argument. + public mutating func edit( + _ body: (inout OutputSpan) throws(E) -> R + ) throws(E) -> R + + /// Grow or shrink the capacity of a {rigid|unique} array instance without discarding + /// its contents. + /// + /// This operation replaces the array's storage buffer with a newly allocated + /// buffer of the specified capacity, moving all existing elements + /// to its new storage. The old storage is then deallocated. + /// + /// - Parameter newCapacity: The desired new capacity. `newCapacity` must be + /// greater than or equal to the current count. + /// + /// - Complexity: O(`count`) + public mutating func reallocate(capacity newCapacity: Int) + + /// Ensure that the array has capacity to store the specified number of + /// elements, by growing its storage buffer if necessary. + /// + /// If `capacity < n`, then this operation reallocates the {rigid|unique} array's + /// storage to grow it; on return, the array's capacity becomes `n`. + /// Otherwise the array is left as is. + /// + /// - Parameter n: The requested number of elements to store. + /// + /// - Complexity: O(`count`) + public mutating func reserveCapacity(_ n: Int) +} + +extension [Rigid|Unique]Array where Element: Copyable { + /// Copy the contents of this array into a newly allocated {rigid|unique} array + /// instance with just enough capacity to hold all its elements. + /// + /// - Complexity: O(`count`) + public func clone() -> Self + + /// Copy the contents of this array into a newly allocated {rigid|unique} array + /// instance with the specified capacity. + /// + /// - Parameter capacity: The desired capacity of the resulting {rigid|unique} array. + /// `capacity` must be greater than or equal to `count`. + /// + /// - Complexity: O(`count`) + public func clone(capacity: Int) -> Self +} +``` + +#### Initializers + +```swift +extension [Rigid|Unique]Array where Element: ~Copyable { + /// Initializes a new {rigid|unique} array with zero capacity and no elements. + /// + /// - Complexity: O(1) + public init() + + /// Initializes a new {rigid|unique} array with the specified capacity and no elements. + public init(capacity: Int) + + /// Creates a new array with the specified capacity, directly initializing + /// its storage using an output span. + /// + /// - Parameters: + /// - capacity: The storage capacity of the new array. + /// - initializer: A callback that gets called at most once to directly + /// populate newly reserved storage within the array. The function + /// is allowed to add fewer than `capacity` items. The array is + /// initialized with however many items the callback adds to the + /// output span before it returns (or before it throws an error). + public init( + capacity: Int, + initializingWith initializer: (inout OutputSpan) throws(E) -> Void + ) throws(E) +} + +extension [Rigid|Unique]Array where Element: Copyable { + /// Creates a new array containing the specified number of a single, + /// repeated value. + /// + /// - Parameters: + /// - repeatedValue: The element to repeat. + /// - count: The number of times to repeat the value passed in the + /// `repeating` parameter. `count` must be zero or greater. + /// + /// - Complexity: O(`count`) + public init(repeating repeatedValue: Element, count: Int) + + /// Creates a new array with the specified capacity, holding a copy + /// of the contents of the given span. + /// + /// - Parameters: + /// - capacity: The storage capacity of the new array, or nil to allocate + /// just enough capacity to store the contents of the span. + /// - span: The span whose contents to copy into the new array. + /// The span must not contain more than `capacity` elements. + public init( + capacity: Int? = nil, + copying span: Span + ) +} +``` + +#### Collection based API + +```swift +extension [Rigid|Unique]Array where Element: ~Copyable { + /// A type that represents a position in the array: an integer offset from the + /// start. + /// + /// Valid indices consist of the position of every element and a "past the + /// end” position that’s not valid for use as a subscript argument. + public typealias Index = Int + + /// A Boolean value indicating whether this array contains no elements. + /// + /// - Complexity: O(1) + public var isEmpty: Bool { + get + } + + /// The number of elements in this array. + /// + /// - Complexity: O(1) + public var count: Int { + get + } + + /// The position of the first element in a nonempty array. This is always zero. + /// + /// - Complexity: O(1) + public var startIndex: Int { + get + } + + /// The array’s "past the end” position—that is, the position one greater than + /// the last valid subscript argument. This is always equal to the array's + /// count. + /// + /// - Complexity: O(1) + public var endIndex: Int { + get + } + + /// The range of indices that are valid for subscripting the array. + /// + /// - Complexity: O(1) + public var indices: Range { + get + } + + /// Accesses the element at the specified position. + /// + /// - Parameter position: The position of the element to access. + /// The position must be a valid index of the array that is not equal + /// to the `endIndex` property. + /// + /// - Complexity: O(1) + public subscript(position: Int) -> Element { + borrow + + mutate + } + + /// Exchanges the values at the specified indices of the array. + /// + /// Both parameters must be valid indices of the array and not equal to + /// endIndex. Passing the same index as both `i` and `j` has no effect. + /// + /// - Parameter i: The index of the first value to swap. + /// - Parameter j: The index of the second value to swap. + /// + /// - Complexity: O(1) + public mutating func swapAt(_ i: Int, _ j: Int) + + /// Returns the position immediately after the given index. + /// + /// - Note: To improve performance, this method does not validate that the + /// index is valid before incrementing it. Index validation is + /// deferred until the resulting index is used to access an element. + /// This optimization may be removed in future versions; do not rely on it. + /// + /// - Parameter index: A valid index of the array. `index` must be less + /// than `endIndex`. + /// - Returns: The index immediately following `index`. + /// - Complexity: O(1) + public func index(after index: Int) -> Int + + /// Returns the position immediately before the given index. + /// + /// - Note: To improve performance, this method does not validate that the + /// index is valid before decrementing it. Index validation is + /// deferred until the resulting index is used to access an element. + /// This optimization may be removed in future versions; do not rely on it. + /// + /// - Parameter index: A valid index of the array. `index` must be greater + /// than `startIndex`. + /// - Returns: The index immediately preceding `index`. + /// - Complexity: O(1) + public func index(before index: Int) -> Int + + /// Replaces the given index with its successor. + /// + /// - Note: To improve performance, this method does not validate that the + /// given index is valid before incrementing it. Index validation is + /// deferred until the resulting index is used to access an element. + /// This optimization may be removed in future versions; do not rely on it. + /// + /// - Parameter index: A valid index of the array. `index` must be less + /// than `endIndex`. + /// - Complexity: O(1) + public func formIndex(after index: inout Int) + + /// Replaces the given index with its predecessor. + /// + /// - Note: To improve performance, this method does not validate that the + /// given index is valid before decrementing it. Index validation is + /// deferred until the resulting index is used to access an element. + /// This optimization may be removed in future versions; do not rely on it. + /// + /// - Parameter index: A valid index of the array. `index` must be greater than + /// `startIndex`. + /// - Complexity: O(1) + public func formIndex(before index: inout Int) + + /// Returns an index that is the specified distance from the given index. + /// + /// The value passed as `n` must not offset `index` beyond the bounds of the + /// array. + /// + /// - Note: To improve performance, this method does not validate that the + /// given index is valid before offsetting it. Index validation is + /// deferred until the resulting index is used to access an element. + /// This optimization may be removed in future versions; do not rely on it. + /// + /// - Parameter index: A valid index of the array. + /// - Parameter n: The distance by which to offset `index`. + /// - Returns: An index offset by `n` from `index`. If `n` is positive, + /// this is the same value as the result of `n` calls to `index(after:)`. + /// If `n` is negative, this is the same value as the result of `abs(n)` + /// calls to `index(before:)`. + /// - Complexity: O(1) + public func index(_ index: Int, offsetBy n: Int) -> Int + + /// Returns the distance between two indices. + /// + /// - Note: To improve performance, this method does not validate that the + /// given index is valid before offsetting it. Index validation is + /// deferred until the resulting index is used to access an element. + /// This optimization may be removed in future versions; do not rely on it. + /// + /// - Parameter start: A valid index of the collection. + /// - Parameter end: Another valid index of the collection. If end is equal + /// to start, the result is zero. + /// - Returns: The distance between `start` and `end`. + /// - Complexity: O(1) + public func distance(from start: Index, to end: Index) -> Int + + /// Offsets the given index by the specified distance, but no further than + /// the given limiting index. + /// + /// If the operation was able to offset `index` by exactly the requested + /// number of steps without hitting `limit`, then on return `n` is set to `0`, + /// and `index` is set to the adjusted index. + /// + /// If the operation hits the limit before it can take the requested number + /// of steps, then on return `index` is set to `limit`, and `n` is set + /// to the number of steps that couldn't be taken. + /// + /// The value passed as `n` must not offset `index` beyond the bounds of the + /// container, unless the index passed as `limit` prevents offsetting beyond + /// those bounds. + /// + /// - Note: To improve performance, this method does not validate that the + /// given index is valid before offsetting it. Index validation is + /// deferred until the resulting index is used to access an element. + /// This optimization may be removed in future versions; do not rely on it. + /// + /// - Parameter index: A valid index of the array. On return, `index` is + /// set to the resulting position. + /// - Parameter n: The distance to offset `index`. + /// On return, `n` is set to zero if the operation succeeded without + /// hitting the limit; otherwise, `n` reflects the number of steps that + /// couldn't be taken. + /// - Parameter limit: A valid index of the array to use as a limit. + /// If `n > 0`, a limit that is less than `index` has no effect. + /// Likewise, if `n < 0`, a limit that is greater than `index` has no + /// effect. + /// - Complexity: O(1) + public func formIndex( + _ index: inout Index, + offsetBy n: inout Int, + limitedBy limit: Index + ) +} +``` + +#### Appends + +```swift +extension [Rigid|Unique]Array where Element: ~Copyable { + /// Adds an element to the end of the array. + /// + /// If the rigid array does not have sufficient capacity to hold any more + /// elements, then this triggers a runtime error. + /// + /// If the unique array does not have sufficient capacity to hold any more + /// elements, then this reallocates the array's storage to grow its capacity, + /// using a geometric growth rate. + /// + /// - Parameter item: The element to append to the array. + /// + /// - Complexity: O(1) + public mutating func append(_ item: consuming Element) + + /// Append a given number of items to the end of this array by populating + /// an output span. + /// + /// If the rigid array does not have sufficient capacity to accommodate the + /// specified number of new items, then this triggers a runtime error. + /// + /// If the unique array does not have sufficient capacity to hold the requested + /// number of new elements, then this reallocates the array's storage to + /// grow its capacity, using a geometric growth rate. + /// + /// If the callback fails to fully populate its output span or if + /// it throws an error, then the array keeps all items that were + /// successfully initialized before the callback terminated the operation. + /// + /// - Parameters: + /// - newItemCount: The number of items to append to the array. + /// - initializer: A callback that gets called at most once to directly + /// populate newly reserved storage within the array. The function + /// is allowed to initialize fewer than `newItemCount` items. + /// The array is extended by however many items the callback appends to + /// the output span before it returns (or throws an error). + /// + /// - Complexity: O(`newItemCount`) + public mutating func append( + addingCount newItemCount: Int, + initializingWith initializer: (inout OutputSpan) throws(E) -> Void + ) throws(E) + + /// Moves the elements of a buffer to the end of this array, leaving the + /// buffer uninitialized. + /// + /// If the rigid array does not have sufficient capacity to hold all items in + /// the buffer, then this triggers a runtime error. + /// + /// If the unique array does not have sufficient capacity to hold all items in + /// the buffer, then this reallocates the array's storage to grow its capacity, + /// using a geometric growth rate. + /// + /// - Parameters: + /// - items: A fully initialized buffer whose contents to move into + /// the array. + /// + /// - Complexity: O(`items.count`) + public mutating func append( + moving items: UnsafeMutableBufferPointer + ) + + /// Moves the elements of an output span to the end of this array, leaving the + /// span empty. + /// + /// If the rigid array does not have sufficient capacity to hold all items + /// from the span in its storage, then this triggers a runtime error. + /// + /// If the unique array does not have sufficient capacity to hold all new items, + /// then this reallocates the array's storage to grow its capacity, + /// using a geometric growth rate. + /// + /// - Parameters: + /// - items: An output span whose contents need to be appended to this array. + /// + /// - Complexity: O(`items.count`) + public mutating func append( + moving items: inout OutputSpan + ) +} + +extension [Rigid|Unique]Array where Element: Copyable { + /// Copies the elements of a buffer to the end of this array. + /// + /// If the rigid array does not have sufficient capacity to hold all items + /// from the buffer in its storage, then this triggers a runtime error. + /// + /// If the unique array does not have sufficient capacity to hold all items + /// in the source buffer, then this automatically grows the array's + /// capacity, using a geometric growth rate. + /// + /// - Parameters: + /// - newElements: A fully initialized buffer whose contents to copy into + /// the array. + /// + /// - Complexity: O(`newElements.count`) when amortized over many + /// invocations on the same array. + public mutating func append( + copying newElements: UnsafeBufferPointer + ) + + /// Copies the elements of a buffer to the end of this array. + /// + /// If the rigid array does not have sufficient capacity to hold all items in + /// the buffer, then this triggers a runtime error. + /// + /// If the unique array does not have sufficient capacity to hold enough elements, + /// then this reallocates the array's storage to extend its capacity, using + /// a geometric growth rate. + /// + /// - Parameters: + /// - newElements: A fully initialized buffer whose contents to copy into + /// the array. + /// + /// - Complexity: O(`newElements.count`) when amortized over many + /// invocations on the same array. + public mutating func append( + copying newElements: UnsafeMutableBufferPointer + ) + + /// Copies the elements of a span to the end of this array. + /// + /// If the rigid array does not have sufficient capacity to hold all items + /// from the span in its storage, then this triggers a runtime error. + /// + /// If the unique array does not have sufficient capacity to hold enough elements, + /// then this reallocates the array's storage to extend its capacity, using a + /// geometric growth rate. + /// + /// - Parameters: + /// - newElements: A span whose contents to copy into the array. + /// + /// - Complexity: O(`newElements.count`) when amortized over many + /// invocations on the same array. + public mutating func append(copying newElements: Span) + + /// Copies the elements of a sequence to the end of this array. + /// + /// If the rigid array does not have sufficient capacity to hold all items + /// from the sequence in its storage, then this triggers a runtime error. + /// + /// If the unique array does not have sufficient capacity to hold enough elements, + /// then this reallocates the array's storage to extend its capacity, using + /// a geometric growth rate. If the input sequence does not provide a precise + /// estimate of its count, then the array's storage may need to be resized + /// more than once. + /// + /// - Parameters: + /// - newElements: The new elements to copy into the array. + /// + /// - Complexity: O(*m*), where *m* is the length of `newElements`, when + /// amortized over many invocations over the same array. + public mutating func append(copying newElements: some Sequence) +} +``` + +#### Insertions + +```swift +extension [Rigid|Unique]Array where Element: ~Copyable { + /// Inserts a new element into the array at the specified position. + /// + /// If the rigid array does not have sufficient capacity to hold any more elements, + /// then this triggers a runtime error. + /// + /// If the unique array does not have sufficient capacity to hold any more elements, + /// then this reallocates storage to extend its capacity, using a geometric + /// growth rate. + /// + /// The new element is inserted before the element currently at the specified + /// index. If you pass the array's `endIndex` as the `index` parameter, then + /// the new element is appended to the container. + /// + /// All existing elements at or following the specified position are moved to + /// make room for the new item. + /// + /// - Parameter item: The new element to insert into the array. + /// - Parameter index: The position at which to insert the new element. + /// `index` must be a valid index in the array. + /// + /// - Complexity: O(`self.count`) + public mutating func insert(_ item: consuming Element, at index: Int) + + /// Inserts a given number of new items into this array at the specified + /// position, using a callback to directly initialize array storage by + /// populating an output span. + /// + /// Existing elements in the array's storage are moved towards the back as + /// needed to make room for the new items. + /// + /// If the capacity of the rigid array isn't sufficient to accommodate the specified + /// number of new elements, then this method triggers a runtime error. + /// + /// If the unique array does not have sufficient capacity to hold the new elements, + /// then this operation reallocates storage to extend its capacity, using a + /// geometric growth rate. + /// + /// var buffer = RigidArray(capacity: 20) + /// buffer.append(copying: [-999, 999]) + /// var i = 0 + /// buffer.insert(addingCount: 3, at: 1) { target in + /// while !target.isFull { + /// target.append(i) + /// i += 1 + /// } + /// } + /// // `buffer` now contains [-999, 0, 1, 2, 999] + /// + /// If the callback fails to fully populate its output span or if + /// it throws an error, then the array keeps all items that were + /// successfully initialized before the callback terminated the insertion. + /// + /// Partial insertions create a gap in array storage that needs to be + /// closed by moving already inserted items to their correct positions given + /// the adjusted count. This adds some overhead compared to adding exactly as + /// many items as promised. + /// + /// - Parameters: + /// - newItemCount: The maximum number of items to insert into the array. + /// - index: The position at which to insert the new items. + /// `index` must be a valid index in the array. + /// - initializer: A callback that gets called at most once to directly + /// populate newly reserved storage within the array. The function + /// is always called with an empty output span. + /// + /// - Complexity: O(`self.count` + `newItemCount`) in addition to the complexity + /// of the callback invocations. + public mutating func insert( + addingCount newItemCount: Int, + at index: Int, + initializingWith initializer: (inout OutputSpan) throws(E) -> Void + ) throws(E) + + /// Moves the elements of a fully initialized buffer into this array, + /// starting at the specified position, and leaving the buffer + /// uninitialized. + /// + /// All existing elements at or following the specified position are moved to + /// make room for the new items. + /// + /// If the capacity of the rigid array isn't sufficient to accommodate the new + /// elements, then this method triggers a runtime error. + /// + /// If the unique array does not have sufficient capacity to hold enough elements, + /// then this reallocates the array's storage to extend its capacity, using a + /// geometric growth rate. + /// + /// - Parameters: + /// - items: A fully initialized buffer whose contents to move into + /// the array. + /// - index: The position at which to insert the new items. + /// `index` must be a valid index in the array. + /// + /// - Complexity: O(`self.count` + `items.count`) + public mutating func insert( + moving items: UnsafeMutableBufferPointer, + at index: Int + ) + + /// Moves the elements of an output span into this array, + /// starting at the specified position, and leaving the span empty. + /// + /// All existing elements at or following the specified position are moved to + /// make room for the new items. + /// + /// If the capacity of the rigid array isn't sufficient to accommodate the new + /// elements, then this method triggers a runtime error. + /// + /// If the unique array does not have sufficient capacity to hold enough elements, + /// then this reallocates the array's storage to extend its capacity, using a + /// geometric growth rate. + /// + /// - Parameters: + /// - items: An output span whose contents to move into + /// the array. + /// - index: The position at which to insert the new items. + /// `index` must be a valid index in the array. + /// + /// - Complexity: O(`self.count` + `items.count`) + public mutating func insert( + moving items: inout OutputSpan, + at index: Int + ) +} + +extension [Rigid|Unique]Array where Element: Copyable { + /// Copies the elements of a fully initialized buffer pointer into this + /// array at the specified position. + /// + /// The new elements are inserted before the element currently at the + /// specified index. If you pass the array’s `endIndex` as the `index` + /// parameter, then the new elements are appended to the end of the array. + /// + /// All existing elements at or following the specified position are moved to + /// make room for the new item. + /// + /// If the capacity of the rigid array isn't sufficient to accommodate the new + /// elements, then this method triggers a runtime error. + /// + /// If the `UniqueArray` does not have sufficient capacity to accommodate the + /// new elements, then this reallocates the array's storage to extend its + /// capacity, using a geometric growth rate. + /// + /// - Parameters: + /// - newElements: The new elements to insert into the array. The buffer + /// must be fully initialized. + /// - index: The position at which to insert the new elements. It must be + /// a valid index of the array. + /// + /// - Complexity: O(`self.count` + `newElements.count`) + public mutating func insert( + copying newElements: UnsafeBufferPointer, at index: Int + ) + + /// Copies the elements of a fully initialized buffer pointer into this + /// array at the specified position. + /// + /// The new elements are inserted before the element currently at the + /// specified index. If you pass the array’s `endIndex` as the `index` + /// parameter, then the new elements are appended to the end of the array. + /// + /// All existing elements at or following the specified position are moved to + /// make room for the new item. + /// + /// If the capacity of the rigid array isn't sufficient to accommodate the new + /// elements, then this method triggers a runtime error. + /// + /// If the unique array does not have sufficient capacity to hold enough elements, + /// then this reallocates the array's storage to extend its capacity, using a + /// geometric growth rate. + /// + /// - Parameters: + /// - newElements: The new elements to insert into the array. The buffer + /// must be fully initialized. + /// - index: The position at which to insert the new elements. It must be + /// a valid index of the array. + /// + /// - Complexity: O(`self.count` + `newElements.count`) + public mutating func insert( + copying newElements: UnsafeMutableBufferPointer, + at index: Int + ) + + /// Copies the elements of a span into this array at the specified position. + /// + /// The new elements are inserted before the element currently at the + /// specified index. If you pass the array’s `endIndex` as the `index` + /// parameter, then the new elements are appended to the end of the array. + /// + /// All existing elements at or following the specified position are moved to + /// make room for the new item. + /// + /// If the capacity of the rigid array isn't sufficient to accommodate the new + /// elements, then this method triggers a runtime error. + /// + /// If the unique array does not have sufficient capacity to hold enough elements, + /// then this reallocates the array's storage to extend its capacity, using a + /// geometric growth rate. + /// + /// - Parameters: + /// - newElements: The new elements to insert into the array. + /// - index: The position at which to insert the new elements. It must be + /// a valid index of the array. + /// + /// - Complexity: O(`self.count` + `newElements.count`) + public mutating func insert( + copying newElements: Span, at index: Int + ) + + /// Copies the elements of a collection into this array at the specified + /// position. + /// + /// The new elements are inserted before the element currently at the + /// specified index. If you pass the array’s `endIndex` as the `index` + /// parameter, then the new elements are appended to the end of the array. + /// + /// All existing elements at or following the specified position are moved + /// to make room for the new item. + /// + /// If the capacity of the rigid array isn't sufficient to accommodate the new + /// elements, then this method triggers a runtime error. + /// + /// If the unique array does not have sufficient capacity to hold enough elements, + /// then this reallocates the array's storage to extend its capacity, using a + /// geometric growth rate. + /// + /// - Parameters: + /// - newElements: The new elements to insert into the array. + /// - index: The position at which to insert the new elements. It must be + /// a valid index of the array. + /// + /// - Complexity: O(`self.count` + `newElements.count`) + public mutating func insert( + copying newElements: some Collection, at index: Int + ) +} +``` + +#### Removals + +```swift +extension [Rigid|Unique]Array where Element: ~Copyable { + /// Removes and returns the last element of the array, if there is one. + /// + /// - Returns: The last element of the array if the array is not empty; + /// otherwise, `nil`. + /// + /// - Complexity: O(1) + public mutating func popLast() -> Element? + + /// Removes and returns the last element of the array. + /// + /// The array must not be empty. + /// + /// - Returns: The last element of the original array. + /// + /// - Complexity: O(1) + public mutating func removeLast() -> Element + + /// Removes and discards the specified number of elements from the end of the + /// array. + /// + /// Attempting to remove more elements than exist in the array triggers a + /// runtime error. + /// + /// - Parameter k: The number of elements to remove from the array. + /// `k` must be greater than or equal to zero and must not exceed + /// the count of the array. + /// + /// - Complexity: O(`k`) + public mutating func removeLast(_ k: Int) + + /// Removes and returns the element at the specified position. + /// + /// All the elements following the specified position are moved to close the + /// gap. + /// + /// - Parameter index: The position of the element to remove. `index` must be + /// a valid index of the array that is not equal to the end index. + /// - Returns: The removed element. + /// + /// - Complexity: O(`self.count`) + public mutating func remove(at index: Int) -> Element + + /// Removes the specified subrange of elements from the array. + /// + /// All the elements following the specified subrange are moved to close the + /// resulting gap. + /// + /// - Parameter bounds: The subrange of the array to remove. The bounds + /// of the range must be valid indices of the array. + /// + /// - Complexity: O(`self.count`) + public mutating func removeSubrange(_ bounds: Range) + + /// Removes the specified subrange of elements from the array. + /// + /// - Parameter bounds: The subrange of the array to remove. The bounds + /// of the range must be valid indices of the array. + /// + /// - Complexity: O(`self.count`) + public mutating func removeSubrange(_ bounds: some RangeExpression) +} +``` + +#### Replacements + +```swift +extension [Rigid|Unique]Array where Element: ~Copyable { + /// Replaces the specified range of elements by a given count of new items, + /// using a callback to directly initialize array storage by populating + /// an output span. + /// + /// The number of new items need not match the number of elements being + /// removed. + /// + /// This method has the same overall effect as calling + /// + /// try array.removeSubrange(subrange) + /// try array.insert( + /// addingCount: newItemCount, + /// at: subrange.lowerBound, + /// initializingWith: initializer) + /// + /// However, it performs faster (by a constant factor) by avoiding moving + /// some items in the array twice. + /// + /// If the rigid array does not have sufficient capacity to accommodate the new + /// elements, then this method triggers a runtime error. + /// + /// If the unique array does not have sufficient capacity to perform the replacement, + /// then this reallocates storage to extend its capacity, using a geometric + /// growth rate. + /// + /// If the callback fails to fully populate its output span or if + /// it throws an error, then the array keeps all items that were + /// successfully initialized before the callback terminated the replacement. + /// + /// Partial replacements create a gap in array storage that needs to be + /// closed by moving subsequent items to their correct positions given + /// the adjusted count. This adds some overhead compared to adding exactly as + /// many items as promised. + /// + /// - Parameters: + /// - subrange: The subrange of the array to replace. The bounds of + /// the range must be valid indices in the array. + /// - newItemCount: The maximum number of new items to insert in place of the old subrange. + /// - initializer: A callback that gets called at most once to directly + /// populate newly reserved storage within the array. The function + /// is always called with an empty output span. + /// + /// - Complexity: O(`self.count` + `newItemCount`) in addition to the complexity + /// of the callback invocations. + public mutating func replaceSubrange( + _ subrange: Range, + addingCount newItemCount: Int, + initializingWith initializer: (inout OutputSpan) throws(E) -> Void + ) throws(E) -> Void + + /// Replaces the specified range of elements by moving the elements of a + /// fully initialized buffer into their place. On return, the buffer is left + /// in an uninitialized state. + /// + /// This method has the effect of removing the specified range of elements + /// from the array and inserting the new elements starting at the same + /// location. The number of new elements need not match the number of elements + /// being removed. + /// + /// If the capacity of the rigid array isn't sufficient to accommodate the new + /// elements, then this method triggers a runtime error. + /// + /// If the unique array does not have sufficient capacity to perform the replacement, + /// then this reallocates the array's storage to extend its capacity, using a + /// geometric growth rate. + /// + /// If you pass a zero-length range as the `subrange` parameter, this method + /// inserts the elements of `newElements` at `subrange.lowerBound`. Calling + /// the `insert(copying:at:)` method instead is preferred in this case. + /// + /// Likewise, if you pass a zero-length buffer as the `newElements` + /// parameter, this method removes the elements in the given subrange + /// without replacement. Calling the `removeSubrange(_:)` method instead is + /// preferred in this case. + /// + /// - Parameters: + /// - subrange: The subrange of the array to replace. The bounds of + /// the range must be valid indices in the array. + /// - newElements: A fully initialized buffer whose contents to move into + /// the array. + /// + /// - Complexity: O(`self.count` + `newElements.count`) + public mutating func replaceSubrange( + _ subrange: Range, + moving newElements: UnsafeMutableBufferPointer + ) + + /// Replaces the specified range of elements by moving the contents of an + /// output span into their place. On return, the span is left empty. + /// + /// This method has the effect of removing the specified range of elements + /// from the array and inserting the new elements starting at the same + /// location. The number of new elements need not match the number of elements + /// being removed. + /// + /// If the capacity of the rigid array isn't sufficient to accommodate the new + /// elements, then this method triggers a runtime error. + /// + /// If the unique array does not have sufficient capacity to perform the replacement, + /// then this reallocates the array's storage to extend its capacity, using a + /// geometric growth rate. + /// + /// If you pass a zero-length range as the `subrange` parameter, this method + /// inserts the elements of `items` at `subrange.lowerBound`. Calling + /// the `insert(moving:at:)` method instead is preferred in this case. + /// + /// Likewise, if you pass a zero-length span as the `items` + /// parameter, this method removes the elements in the given subrange + /// without replacement. Calling the `removeSubrange(_:)` method instead is + /// preferred in this case. + /// + /// - Parameters: + /// - subrange: The subrange of the array to replace. The bounds of + /// the range must be valid indices in the array. + /// - items: An output span whose contents are to be moved into the array. + /// + /// - Complexity: O(`self.count` + `items.count`) + public mutating func replaceSubrange( + _ subrange: Range, + moving items: inout OutputSpan + ) +} + +extension [Rigid|Unique]Array where Element: Copyable { + /// Replaces the specified subrange of elements by copying the elements of + /// the given buffer pointer, which must be fully initialized. + /// + /// This method has the effect of removing the specified range of elements + /// from the array and inserting the new elements starting at the same location. + /// The number of new elements need not match the number of elements being + /// removed. + /// + /// If the capacity of the rigid array isn't sufficient to accommodate the new + /// elements, then this method triggers a runtime error. + /// + /// If the capacity of the unique array isn't sufficient to perform the replacement, + /// then this reallocates the array's storage to extend its capacity, using a + /// geometric growth rate. + /// + /// If you pass a zero-length range as the `subrange` parameter, this method + /// inserts the elements of `newElements` at `subrange.lowerBound`. Calling + /// the `insert(copying:at:)` method instead is preferred in this case. + /// + /// Likewise, if you pass a zero-length buffer as the `newElements` + /// parameter, this method removes the elements in the given subrange + /// without replacement. Calling the `removeSubrange(_:)` method instead is + /// preferred in this case. + /// + /// - Parameters: + /// - subrange: The subrange of the array to replace. The bounds of + /// the range must be valid indices in the array. + /// - newElements: The new elements to copy into the collection. + /// + /// - Complexity: O(*n* + *m*), where *n* is count of this array and + /// *m* is the count of `newElements`. + public mutating func replaceSubrange( + _ subrange: Range, + copying newElements: UnsafeBufferPointer + ) + + /// Replaces the specified subrange of elements by copying the elements of + /// the given buffer pointer, which must be fully initialized. + /// + /// This method has the effect of removing the specified range of elements + /// from the array and inserting the new elements starting at the same location. + /// The number of new elements need not match the number of elements being + /// removed. + /// + /// If the capacity of the rigid array isn't sufficient to accommodate the new + /// elements, then this method triggers a runtime error. + /// + /// If the capacity of the unique array isn't sufficient to perform the replacement, + /// then this reallocates the array's storage to extend its capacity, using a + /// geometric growth rate. + /// + /// If you pass a zero-length range as the `subrange` parameter, this method + /// inserts the elements of `newElements` at `subrange.lowerBound`. Calling + /// the `insert(copying:at:)` method instead is preferred in this case. + /// + /// Likewise, if you pass a zero-length buffer as the `newElements` + /// parameter, this method removes the elements in the given subrange + /// without replacement. Calling the `removeSubrange(_:)` method instead is + /// preferred in this case. + /// + /// - Parameters: + /// - subrange: The subrange of the array to replace. The bounds of + /// the range must be valid indices in the array. + /// - newElements: The new elements to copy into the collection. + /// + /// - Complexity: O(*n* + *m*), where *n* is count of this array and + /// *m* is the count of `newElements`. + public mutating func replaceSubrange( + _ subrange: Range, + copying newElements: UnsafeMutableBufferPointer + ) + + /// Replaces the specified subrange of elements by copying the elements of + /// the given span. + /// + /// This method has the effect of removing the specified range of elements + /// from the array and inserting the new elements starting at the same location. + /// The number of new elements need not match the number of elements being + /// removed. + /// + /// If the capacity of the rigid array isn't sufficient to accommodate the new + /// elements, then this method triggers a runtime error. + /// + /// If the capacity of the unique array isn't sufficient to perform the replacement, + /// then this reallocates the array's storage to extend its capacity, using a + /// geometric growth rate. + /// + /// If you pass a zero-length range as the `subrange` parameter, this method + /// inserts the elements of `newElements` at `subrange.lowerBound`. Calling + /// the `insert(copying:at:)` method instead is preferred in this case. + /// + /// Likewise, if you pass a zero-length span as the `newElements` + /// parameter, this method removes the elements in the given subrange + /// without replacement. Calling the `removeSubrange(_:)` method instead is + /// preferred in this case. + /// + /// - Parameters: + /// - subrange: The subrange of the array to replace. The bounds of + /// the range must be valid indices in the array. + /// - newElements: The new elements to copy into the collection. + /// + /// - Complexity: O(*n* + *m*), where *n* is count of this array and + /// *m* is the count of `newElements`. + public mutating func replaceSubrange( + _ subrange: Range, + copying newElements: Span + ) + + /// Replaces the specified subrange of elements by copying the elements of + /// the given collection. + /// + /// This method has the effect of removing the specified range of elements + /// from the array and inserting the new elements starting at the same location. + /// The number of new elements need not match the number of elements being + /// removed. + /// + /// If the capacity of the rigid array isn't sufficient to accommodate the new + /// elements, then this method triggers a runtime error. + /// + /// If the capacity of the unique array isn't sufficient to perform the replacement, + /// then this reallocates the array's storage to extend its capacity, using a + /// geometric growth rate. + /// + /// If you pass a zero-length range as the `subrange` parameter, this method + /// inserts the elements of `newElements` at `subrange.lowerBound`. Calling + /// the `insert(copying:at:)` method instead is preferred in this case. + /// + /// Likewise, if you pass a zero-length collection as the `newElements` + /// parameter, this method removes the elements in the given subrange + /// without replacement. Calling the `removeSubrange(_:)` method instead is + /// preferred in this case. + /// + /// - Parameters: + /// - subrange: The subrange of the array to replace. The bounds of + /// the range must be valid indices in the array. + /// - newElements: The new elements to copy into the collection. + /// + /// - Complexity: O(*n* + *m*), where *n* is count of this array and + /// *m* is the count of `newElements`. + public mutating func replaceSubrange( + _ subrange: Range, + copying newElements: consuming some Collection + ) +} +``` + +#### Conformances + +```swift +extension [Rigid|Unique]Array: Equatable where Element: Equatable & ~Copyable { + public static func ==(left: borrowing Self, right: borrowing Self) -> Bool +} + +extension [Rigid|Unique]Array: Hashable where Element: Hashable & ~Copyable { + public func hash(into hasher: inout Hasher) +} + +extension [Rigid|Unique]Array: CustomStringConvertible where Element: ~Copyable { + public var description: String { get } +} + +extension [Rigid|Unique]Array: CustomDebugStringConvertible where Element: ~Copyable { + public var debugDescription: String { get } +} + +extension [Rigid|Unique]Array: BorrowingSequence where Element: ~Copyable { + @lifetime(borrow self) + public func makeBorrowingIterator() -> SpanIterator +} +``` + +Note that these array types will conform to the newly introduced +`BorrowingSequence` proposed in [SE-0516]. They will use the `SpanIterator` +defined in that proposal as their iterators. + +[SE-0516]: https://github.com/swiftlang/swift-evolution/blob/main/proposals/0516-borrowing-sequence.md + +While this proposal lists conformances to `CustomStringConvertible` and +`CustomDebugStringConvertible`, these conformances can only be shipped once +[SE-0499] gets implemented. Meanwhile, the types still provide (for now, rudimentary) +implementations of the `description` and `debugDescription` properties. + +[SE-0499]: https://github.com/swiftlang/swift-evolution/blob/main/proposals/0499-support-non-copyable-simple-protocols.md + +### API _only_ on `RigidArray` + +#### Basics + +```swift +extension RigidArray where Element: ~Copyable { + /// A Boolean value indicating whether this rigid array is fully populated. + /// If this property returns true, then the array's storage is at capacity, + /// and it cannot accommodate any additional elements. + /// + /// - Complexity: O(1) + public var isFull: Bool { + get + } +} +``` + +#### Initializers + +```swift +extension RigidArray where Element: ~Copyable { + /// Creates a new array with the specified capacity, holding a copy + /// of the contents of a given sequence. + /// + /// - Parameters: + /// - capacity: The storage capacity of the new array. + /// - contents: The sequence whose contents to copy into the new array. + /// The sequence must not contain more than `capacity` elements. + public init( + capacity: Int, + copying contents: some Sequence + ) + + /// Creates a new array with the specified capacity, holding a copy + /// of the contents of a given collection. + /// + /// - Parameters: + /// - capacity: The storage capacity of the new array, or nil to allocate + /// just enough capacity to store the contents. + /// - contents: The collection whose contents to copy into the new array. + /// The collection must not contain more than `capacity` elements. + public init( + capacity: Int? = nil, + copying contents: some Collection + ) +} +``` + +#### Appends + +```swift +extension RigidArray where Element: ~Copyable { + /// Adds an element to the end of the array, if possible. + /// + /// If the array does not have sufficient capacity to hold any more elements, + /// then this returns the given item without appending it; otherwise it + /// returns nil. + /// + /// - Parameter item: The element to append to the array. + /// - Returns: `item` if the array is full; otherwise nil. + /// + /// - Complexity: O(1) + public mutating func pushLast(_ item: consuming Element) -> Element? +} +``` + +#### Removals + +```swift +extension RigidArray where Element: ~Copyable { + /// Removes all elements from the array, preserving its allocated capacity. + /// + /// - Complexity: O(*n*), where *n* is the original count of the array. + public mutating func removeAll() +} +``` + +### API _only_ on `UniqueArray` + +#### Initializers + +```swift +extension UniqueArray where Element: ~Copyable { + /// Creates a new array with the specified initial capacity, holding a copy + /// of the contents of a given sequence. + /// + /// - Parameters: + /// - capacity: The storage capacity of the new array, or nil to allocate + /// just enough capacity to store the contents. + /// - contents: The sequence whose contents to copy into the new array. + public init( + capacity: Int? = nil, + copying contents: some Sequence + ) + + /// Initializes a new unique array with the specified capacity and no elements. + public init(minimumCapacity: Int) +} +``` + +#### Removals + +```swift +extension UniqueArray where Element: ~Copyable { + /// Removes all elements from the array, optionally preserving its + /// allocated capacity. + /// + /// - Complexity: O(*n*), where *n* is the original count of the array. + public mutating func removeAll(keepingCapacity keepCapacity: Bool = false) +} +``` + + +## Source compatibility + +`RigidArray` and `UniqueArray` are new types within the Standard Library; adding +them is a source compatible change. Developers who currently import these +types from [swift-collections] (or define their own types with the same names), +the imported/custom types will still work, due to the shadowing rule for +Standard Library type names. + +## ABI compatibility + +This proposal is purely additive to the Standard Library's ABI; the addition +does not break any existing binary. + +The types are proposed to be frozen; this prevents future changes to their +representation. (This may be relevant for `UniqueArray`, which may want to +keep track of its reserved capacity to implement shrinking.) + +## Implications on adoption + +`RigidArray` and `UniqueArray` are new types within the Standard Library, so +adopters must use at least the version of Swift that introduced these types. +For developers using these types from +[swift-collections](https://github.com/apple/swift-collections), it may make +sense to continue using those versions of these types due to the backward +deployment nature of having the source from a package. + +## Future directions + +### `Clonable` + +This proposal, like the [`UniqueBox`][UniqueBox] +proposal, introduces `clone()` (and `clone(capacity:)`) for both `RigidArray` +and `UniqueArray`. Clone is an explicit deep copy returning an owned array +instance for the caller. Note that this API currently requires +`Element: Copyable`, but this disallows nested 2d arrays +`UniqueArray>` for example. As mentioned in the `UniqueBox` +proposal, there is room for a potential `Clonable` protocol here that would +enable such functionality: + +[UniqueBox]: https://github.com/swiftlang/swift-evolution/blob/main/proposals/0517-uniquebox.md + +```swift +public protocol Cloneable: ~Copyable { + func clone() -> Self +} +``` + +### Rigid and unique variants of other standard data structures + +The `Rigid` and `Unique` naming prefixes proposed by this proposal are +intended to establish a general naming pattern for container types with similar +behavior. + +The [swift-collections] package +defines `RigidDeque` and `UniqueDeque` types, implementing ring buffers with +the same semantics (and intended target audience) as `RigidArray` and +`UniqueArray`. The package also comes with ownership-aware prototypes of the +standard hashed `Set` and `Dictionary` container types, also using the `Rigid` +and `Unique` prefixes this way. + +`RigidDeque`, `UniqueDeque`, `RigidSet`, `UniqueSet`, `RigidDictionary` and +`UniqueDictionary` are all potential future additions to the Swift Standard +Library. + +### Container protocols + +While this proposal does add the `BorrowingSequence` conformance for both of +the proposed array types, we still aren't ready to propose any container +protocols on top of it. [swift-collections] +is currently [exploring design approaches for such abstractions][containers] + +[containers]: https://github.com/apple/swift-collections/tree/1.4.1/Sources/ContainersPreview/Protocols/Container + +### Literal initialization + +The proposed array types do not conform to `ExpressibleByArrayLiteral`, +regardless of whether the element is copyable or not. The existing protocol is +built around the construction of an `Array` instance through a variadic +initializer; this does not (easily) lend itself to generalization, and forcing +`RigidArray`/`UniqueArray` initialization to go through a temporary `Array` +instance would not satisfy the performance goals of these types, even if +the conformance would be restricted to copyable elements. + +One potentially workable way to reformulate array literal initialization +would be to express it in terms of the in-place initialization +of storage, through populating `OutputSpan` instances over the target type's +storage buffer(s): + +```swift +protocol ArrayLiterable: ~Copyable { + associatedtype ArrayLiteralElement: ~Copyable + + init( + arrayLiteralCount count: Int, + initializingWith initializer: (inout OutputSpan) throws(E) -> Void + ) throws(E) +} +``` + +In this approach, an array initialization expression like `[a, b, c, d]` in +type context `T` would get expanded into something like the following +pseudocode: + +```swift +T(arrayLiteralCount: 4) { target in + target.append(a) + target.append(b) + target.append(c) + target.append(d) +} +``` + +(This assumes that T has contiguous storage. Allow the initialization of potentially +discontiguous target storage is a little trickier, involving an inline array +of item-returning functions.) + +Exploring this or other approaches is expected to be the subject of subsequent +work. + +## Alternatives considered + +### Allocator arguments and similar configuration knobs + +Since we're proposing new array types, we have the unique (pun intended) +opportunity to allow these data structures to be allocated with custom allocators. + +```swift +public struct UniqueArray: ~Copyable {} +``` + +Adding an allocator generic argument is very reminiscent of how container types +work in C++ and Rust. + +This approach would require an `Allocator` protocol that custom allocators could +conform to and provide some `SystemAllocator` that comes by default in the +Standard Library (similar to SystemRandomNumberGenerator): + +```swift +protocol Allocator { + func allocate(_: T.Type) -> UnsafeMutablePointer + + func deallocate(_: UnsafeMutablePointer) + + ... +} +``` + +A similar idea would be for `UniqueArray` to support custom growth/shrink rates +by taking a type argument describing these parameters, perhaps by rolling these +into static property requirements in a refinement of the `Allocator` protocol. + +However, such type arguments make working with these types quite a bit more +awkward: + +```swift +func foo(_ x: borrowing UniqueArray) +// error: generic type 'UniqueArray' specialized with too few type parameters (got 1, but expected 2) +``` + +C++ and Rust solve this particular issue by allowing generic type parameters +to provide default values. So in our original Unique definition we could have: + +```swift +public struct UniqueArray: ~Copyable { + init(allocator: Alloc) { ... } +} + +extension UniqueArray where Allocator == SystemAllocator { + init() { self.init(allocator: SystemAllocator()) } +} +``` + +At first glance, this would let us to work with such types with minimal pain. +However, this assumes the implementation of a major new language feature that +does not currently exist. + +But an even worse issue has to do with the function `foo(_:)` above. With +default type arguments, it would indeed become a valid declaration; but it would +typically overconstrain its parameter by requiring it to use the system +allocator. In fact, that is generally the wrong choice! Functions that borrow +a unique array have no reason to care what allocator it uses, as they have no +way to mutate them anyway. `foo` would need to become generic: + +```swift +func foo(_ x: borrowing UniqueArray) +``` + +So we'd effectively have to spell out the allocator arguments anyway, with the +extra twist of a new undiagnosed issue if we forget to do that. + +For functions that want to consume or mutate arrays, these allocator arguments +would often need to be named and propagated throughout the code base, +polluting interface definitions and obfuscating work. +Indeed, such viral type argument pollution is a frequent complaint of C++ +programmers. + +Generic type configuration parameters would lead to a worse developer +experience when debugging code unless we implement major debugger improvements, +as stack traces would spell out the full type names, including defaulted +arguments. (This is also a frequent issue with C++.) + +The `Allocator` abstraction also raises the question of whether conforming +implementations need to be copyable, whether their allocate/deallocate methods +are marked mutating (and therefore incompatible with concurrent use), and +how exactly we expect Swift programmers to implement allocators, anyway. Perhaps +most crucially, allocators traffic in unsafe pointers -- they would be a very +prominent plot hole in Swift's memory safety story. + +### Making `RigidArray`/`UniqueArray` share their storage representation with `Array` + +As `UniqueArray` is simply a thin wrapper type around a `RigidArray` instance, +their instances are trivially cross-convertible: we can turn a `RigidArray` into +`UniqueArray` (or vice versa) with 𝛩(1) complexity initializers. + +It would be wonderful if the classic `Array` type would also be part of this +cross-convertible family. Unfortunately, `Array`'s storage representation is +part of its ABI, and is not amenable to changes. It would be inappropriate +for `RigidArray`/`UniqueArray` to borrow the same representation, as it was built +around a particular generic class with tail-allocated storage. Using the same +representation would introduce unnecessary performance overhead in the new +low-level types, making them less competitive with similar types in competing +systems programming languages. Therefore, converting an `Array` instance to +one of the new types (or vice versa) requires copying/moving elements to newly +allocated storage in linear time. We do not expect this will be a major issue +in practice. + +### Generalizing `Array` to support noncopyable elements + +The authors consider copy-on-write value semantics to be a major feature of +`Array` and its fellow standard Collection types, and of Swift itself. We +currently believe that the idea of dismantling this feature by allowing `Array` +to become conditionally noncopyable (or otherwise conditionalizing its +copy-on-write behavior) would in fact be working against the goals of Swift's +Ownership Manifesto, and it would be wholly impractical in practice. + +Any such work would also not negate the need for dedicated fixed-capacity and +guaranteed-noncopyable array variants that we propose in this document, either: +it would not be appropriate to force developers to use the dynamically +resizing, copy-on-write `Array` type just because their `Element` happens to be +copyable. There is real, pressing need for a `RigidArray` of integers, or a +`UniqueArray` of floats, whether or not `Array` eventually ends up supporting +noncopyable contents. That said, this proposal does nothing to rule out such +work in the future. + +### Move these types into the default Swift module + +We don't feel as if these types should be included in the default namespace of +Swift programmers because `Array` should still be everyone's first choice. The +inclusion of these array types do not supersede `Array`, but are merely +alternative tools when working in more constrained environments.