# Architecture
Technical decisions and design rationale for SwiftVLC, written for
contributors and reviewers.
> Looking for **how to use** SwiftVLC? See the DocC guides (published on
> Swift Package Index) or run `swift package preview-documentation
> --target SwiftVLC` locally. This document covers the *why* behind the
> design: build/release infrastructure, C-interop patterns, isolation
> choices, deinit ordering, and the testing strategy.
## Contents
- [High-Level Overview](#high-level-overview)
- [Tech Stack](#tech-stack)
- [Module Architecture](#module-architecture)
- [C Interop Layer](#c-interop-layer)
- [Concurrency & Threading Model](#concurrency--threading-model)
- [Event System](#event-system)
- [Memory Management](#memory-management)
- [Video Rendering](#video-rendering)
- [Picture-in-Picture](#picture-in-picture)
- [Error Handling](#error-handling)
- [Testing Strategy](#testing-strategy)
- [Build & Release Infrastructure](#build--release-infrastructure)
- [Project Structure](#project-structure)
---
## High-Level Overview
```mermaid
flowchart TB
subgraph App["SwiftUI App"]
VV["VideoView(player)"]
PS["player.state"]
PV["player.volume"]
end
subgraph SwiftVLC["SwiftVLC Library"]
direction TB
subgraph Top["Primary Modules"]
Player["Player
@Observable · @MainActor"]
Media["Media
Sendable · async"]
Audio["Audio
EQ · Modes"]
Video["Video
VideoView · Adjustments"]
end
subgraph Bottom["Supporting Modules"]
Playlist["Playlist
MediaList · ListPlayer"]
Discovery["Discovery
LAN/UPnP · Renderer"]
PiP["PiP
iOS native drawable or direct vmem · macOS SPI drawable"]
EventBridge["EventBridge
C → AsyncStream"]
end
end
subgraph CLibVLC["CLibVLC"]
Shim["C shim + libVLC 4.0 headers"]
end
subgraph XCF["libvlc.xcframework"]
Binary["Pre-built libVLC 4.0
iOS · macOS · tvOS · visionOS · Catalyst
21 system frameworks · 7 system libraries"]
end
App --> SwiftVLC
Top --> Bottom
SwiftVLC --> CLibVLC
CLibVLC --> XCF
```
**Key concepts:**
- **Swift-first**: Direct C → Swift bindings, no Objective-C intermediary (unlike VLCKit)
- **Observable state**: `@Observable` `@MainActor` Player drives SwiftUI updates automatically
- **Typed concurrency**: Swift 6 strict concurrency. Main-actor types own UI-adjacent C resources, and cross-actor resource/value types are `Sendable`.
- **One-liner rendering**: `VideoView(player)` covers the setup, delegate wiring, and callbacks in one call.
- **Typed errors**: `throws(VLCError)` for compile-time error handling
---
## Tech Stack
| Component | Choice | Why |
|---|---|---|
| **Language** | Swift 6.3+ with Xcode 26.4+ | Strict concurrency, typed throws, `@Observable`, upcoming feature flags |
| **C Bindings** | libVLC 4.0 C API | Direct access, no Objective-C overhead |
| **State** | `@Observable` / `@MainActor` | Automatic SwiftUI integration, thread safety |
| **Events** | `AsyncStream` | Native structured concurrency, multi-consumer |
| **Video** | `UIView` / `NSView` via `set_nsobject` | Platform-native rendering, zero-copy |
| **PiP** | iOS `PiPVideoView`: native drawable; direct controller: vmem → `AVSampleBufferDisplayLayer`; macOS view: native drawable behind SPI | Public iOS PiP, a public direct sample-buffer route, plus explicit private-API macOS opt-in |
| **Thread Safety** | `Mutex`, `Sendable`, `nonisolated(unsafe)` | Compile-time data race prevention |
| **Testing** | Swift Testing framework | Modern `@Test`, `#expect`, tags, traits |
| **Platforms** | iOS 18+, macOS 15+, tvOS 18+, visionOS 2+, Mac Catalyst 18+ | Unified SwiftUI minimum |
---
## Module Architecture
### Core
Foundation types shared across all modules.
| File | Type | Purpose |
|---|---|---|
| `VLCInstance.swift` | `final class VLCInstance: Sendable` | Manages `libvlc_instance_t*` lifecycle. Singleton `shared` or custom with arguments. Owns the per-instance `dialogRegistration` Mutex. |
| `VLCError.swift` | `enum VLCError: Error, Sendable, Equatable, Hashable, LocalizedError, CustomStringConvertible` | Typed errors with hand-rolled per-case accessors (`error.parseTimeout`, `error.mediaCreationFailed`, …). Auto-synthesized `Equatable`/`Hashable` over `String` payloads. |
| `Broadcaster.swift` | `final class Broadcaster` | Internal multi-consumer fan-out used by the dialog, renderer, log, player-event, and playback-intent streams. Exposes `subscribe`, `broadcast`, `finishAll` (allows resubscribe) and `terminate` (permanent — future `subscribe` calls return immediately-finished streams) inside the module. Lifecycle reconciliation runs on a private serial queue. |
| `DialogHandler.swift` | `final class DialogHandler: Sendable` | Bridges libVLC's dialog callbacks (`login`, `question`, `progress`, `error`) onto a `Broadcaster`. `DialogID` carries the dialog handle through `DialogIDStorage` for safe `dismiss()` + post calls. |
| `Logging.swift` | `AsyncStream` via `LogBroadcaster` | Filterable log stream backed by `Broadcaster`. C shim formats `va_list` before Swift callback. `LogNoiseFilter` demotes known-noisy libVLC errors to warnings. |
| `Signposts.swift` | `enum Signposts` | Process-wide `OSSignposter` for `org.swiftvlc` subsystem. Hot paths (`Broadcaster.broadcast`, `Player.handleEvent`, `EventBridge.callback`, `PixelBufferRenderer.outputPixelBuffer`) emit signposts visible in Instruments. Zero cost when no profiler is attached. |
| `Duration+Extensions.swift` | Extensions on `Duration` | `milliseconds`, `microseconds` properties and `formatted` display string |
**Default VLC arguments:** `["--no-video-title-show", "--no-snapshot-preview"]`. `--no-stats` is *not* in the defaults — leaving it on would silently zero every `Media.statistics()` read, which is almost never what a caller wants. On macOS the default keeps VLC's Apple sample-buffer display available for inline playback, including its paired subtitle layer. The private native-drawable PiP backend is present for explicitly opted-in non-App-Store builds, but it is SPI and disabled by default.
### Player
The central observable type that drives all playback.
| File | Type | Purpose |
|---|---|---|
| `Player.swift` | `@Observable @MainActor class` | Wraps `libvlc_media_player_t*`. Lifecycle, drawable management, media loading, and core playback control. Extension files group audio, chapters, events, overlays, programs, recording, typed values, and native-lifecycle helpers. |
| `Player+Events.swift` | `extension Player` | The libVLC event-consumer task and `handleEvent(_:)` dispatch — pause-transition state machine, deferred-pause command, native-state probing, media-derived state reset. |
| `Player+Audio.swift` | `extension Player` | Audio-output device selection, role, mix mode, stereo mode. |
| `Player+Chapters.swift` | `extension Player` | Title/chapter navigation, DVD menu actions. |
| `Player+ABLoop.swift` | `extension Player` | A-B loop state plus checked time/position mutations. |
| `Player+Programs.swift` | `extension Player` | DVB/MPEG-TS program selection, renderer targeting, mid-playback `recast(to:)`, and scrambled-state polling. |
| `Player+Recording.swift` | `extension Player` | Snapshot capture and recording start/stop. |
| `Player+Overlays.swift` | `extension Player` | Scoped `withMarquee`/`withLogo`/`withAdjustments` accessors for the `~Copyable ~Escapable` overlay types. |
| `Player+Typed.swift` | `extension Player` | Typed read-only accessors for raw `position`/`volume`/`rate`/`subtitleTextScale`, plus explicit checked mutation methods such as `setPlaybackRate(_:)`. |
| `PlaybackValues.swift` | 5 typed wrapper structs | `PlaybackPosition`, `Volume`, `PlaybackRate`, `SubtitleScale`, `EqualizerGain`. Each is `Sendable, Hashable, Comparable, ExpressibleByFloatLiteral`, clamps finite input to its valid range, and maps `NaN` to a safe named default. |
| `EventBridge.swift` | `internal class` | C callbacks → `Broadcaster` multi-consumer broadcaster. |
| `PlayerState.swift` | `enum PlayerState` | `.idle`, `.opening`, `.buffering`, `.playing`, `.paused`, `.stopped`, `.stopping`, `.error`. Buffer fill is exposed separately as `Player.bufferFill` so `.paused` players still publish progress. |
| `PlayerEvent.swift` | `enum PlayerEvent` | Typed Swift cases mapped from libVLC's player event types. Hand-rolled per-case accessors (`event.stateChanged`, `event.timeChanged`, …). |
| `PlayerRole.swift` | `enum PlayerRole` | Audio behavior hints: `.music`, `.video`, `.communication`, `.game`, etc. |
| `ABLoop.swift` | `enum ABLoopState` | `.none` → `.pointASet` → `.active` |
| `NavigationAction.swift` | `enum NavigationAction` | DVD/Blu-ray menu: `.activate`, `.up`, `.down`, `.left`, `.right`, `.popup` |
| `Program.swift` | `struct Program` | DVB/MPEG-TS program: id, name, isSelected, isScrambled |
**Player API surface:**
```swift
// Observable properties (auto-update SwiftUI)
player.state // PlayerState
player.currentTime // Duration
player.duration // Duration?
player.isSeekable // Bool
player.isPausable // Bool
player.currentMedia // Media?
player.audioTracks // [Track]
player.videoTracks // [Track]
player.subtitleTracks // [Track]
// Playback observations
player.position // Double (0.0–1.0), read-only
player.volume // Float (0.0–2.0), read-only requested volume shadow
player.rate // Float (current libVLC rate), read-only
player.audioDelay // Duration, read-only
player.subtitleDelay // Duration, read-only
player.subtitleTextScale // Float, read-only
// Mutable properties
player.isMuted // Bool
player.selectedAudioTrack // Track?
player.selectedSubtitleTrack // Track?
player.aspectRatio // AspectRatio
// Checked mutations for read-only playback observations
try player.seek(to: PlaybackPosition(0.5))
try player.setAudioVolume(0.8)
try player.setPlaybackRate(1.5)
try player.setAudioDelay(.milliseconds(30))
try player.setSubtitleDelay(.milliseconds(-100))
player.setSubtitleScale(1.25)
// Playback control
try player.play(url: someURL)
player.pause()
player.resume()
try player.seek(to: .seconds(30))
try player.seek(by: .seconds(-10))
player.stop()
// Advanced
try player.setABLoop(a: .seconds(10), b: .seconds(20))
try player.takeSnapshot(to: path, width: 320, height: 240)
player.startRecording(to: directoryPath)
try player.updateViewpoint(Viewpoint(yaw: 90, pitch: 0, roll: 0, fieldOfView: 80))
```
### Media
Media resource creation, parsing, and metadata.
| File | Type | Purpose |
|---|---|---|
| `Media.swift` | `final class Media: Sendable` | Wraps `libvlc_media_t*`. Create from URL, path, or file descriptor. Async parsing with cancellation. |
| `Metadata.swift` | `struct Metadata: Sendable` | libVLC's metadata keys surfaced as typed properties (title, artist, album, duration, artworkURL, genre, …) |
| `Track.swift` | `struct Track: Sendable` | Audio/video/subtitle track info with type-specific fields (channels, resolution, encoding) |
| `ThumbnailRequest.swift` | Extension on `Media` | `thumbnail(at:width:height:crop:timeout:) async throws → Data` |
| `MediaStatistics.swift` | `struct MediaStatistics: Sendable` | Runtime stats: decoded/displayed/lost frames, bitrates, buffer counts |
**Parsing flow:**
```mermaid
sequenceDiagram
participant App
participant Media
participant libVLC
App->>Media: parse(timeout:) async throws
Media->>libVLC: libvlc_media_parse_request()
libVLC-->>Media: MediaParsedChanged event
alt Task cancelled
Media->>libVLC: libvlc_media_parse_stop()
else Timeout exceeded
Media-->>App: throws .parseTimeout
else Success
Media-->>App: returns Metadata
end
```
### Audio
Audio output, equalization, and channel configuration.
| File | Type | Purpose |
|---|---|---|
| `AudioOutput.swift` | `struct AudioOutput`, `struct AudioDevice` | Available output modules and devices. Extensions on `VLCInstance` and `Player`. |
| `Equalizer.swift` | `@Observable @MainActor class Equalizer` | 10-band EQ with preamp (-20 to +20 dB). libVLC's built-in presets. Attach via `player.equalizer`; mutations re-apply automatically. |
| `AudioChannelMode.swift` | `enum StereoMode`, `enum MixMode` | Stereo/mono/Dolby, 4.0/5.1/7.1/binaural mixing |
### Video
Video rendering, overlays, and adjustments.
| File | Type | Purpose |
|---|---|---|
| `VideoView.swift` | SwiftUI `UIViewRepresentable` / `NSViewRepresentable` | One-liner: `VideoView(player)`. Platform-specific `VideoSurface` underneath. |
| `AspectRatio.swift` | `enum AspectRatio` | `.default`, `.ratio(w, h)`, `.fill` |
| `VideoAdjustments.swift` | `@MainActor struct VideoAdjustments` | brightness, contrast, hue, saturation, gamma — accessed via `player.adjustments` |
| `Marquee.swift` | `@MainActor struct Marquee` | Scrolling text overlay: text, color, opacity, position, timeout |
| `Logo.swift` | `@MainActor struct Logo` | Image overlay: file path, position, opacity, animation |
| `Viewpoint.swift` | `struct Viewpoint: Sendable` | 360° video: yaw, pitch, roll, fieldOfView (degrees) |
### Playlist
Playlist management and sequential/looped playback.
| File | Type | Purpose |
|---|---|---|
| `MediaList.swift` | `final class MediaList: Sendable` | Thread-safe list wrapping `libvlc_media_list_t*`. Append/insert/remove with internal locking. |
| `MediaListPlayer.swift` | `@MainActor class MediaListPlayer` | Sequential playback with `play(at:)`, `next()`, `previous()` |
| `PlaybackMode.swift` | `enum PlaybackMode` | `.default`, `.loop`, `.repeat` |
### Discovery
Network service and renderer discovery.
| File | Type | Purpose |
|---|---|---|
| `MediaDiscoverer.swift` | `final class MediaDiscoverer: Sendable` | Discovers media on LAN/SMB/UPnP/SAP. Returns `MediaList` of found items. |
| `RendererDiscoverer.swift` | `final class RendererDiscoverer: Sendable` | Discovers renderer devices exposed by libVLC plugins. `AsyncStream` for add/remove. |
### PiP (iOS/macOS only)
Picture-in-Picture uses the platform path that best matches libVLC's
video output. `PiPVideoView` uses libVLC's native drawable integration on
iOS. Instantiating `PiPController` directly instead installs public vmem
callbacks and exposes an `AVSampleBufferDisplayLayer`. macOS has a
native-drawable backend behind `PrivateMacOSPiP` SPI because the public
sample-buffer mirror crops incorrectly on supported macOS releases.
| File | Type | Purpose |
|---|---|---|
| `PiPController.swift` | `@MainActor class PiPController` | PiP lifecycle, timebase sync, deferred-pause state machine, native-state observer task. |
| `PiPController+Delegate.swift` | extension + private `PiPPlaybackDelegateProxy` | `AVPictureInPictureControllerDelegate` conformance plus the sample-buffer playback delegate proxy that breaks AVKit's strong-retain cycle (AVKit captures the playback delegate strongly even though the header declares it `weak`). Also hosts the `pipMainActorSync` helper for non-main AVKit callbacks that need synchronous main-actor answers. |
| `PiPController+AudioSession.swift` | extension | Configures the iOS playback category at construction, defers activation until playback/PiP intent, and retries after transient activation failure. |
| `PiPController+PlaybackState.swift` | extension | Payload-driven AVKit range invalidation and seekability-to-linear-playback policy that is independent of `Player` subscriber ordering. |
| `PiPControllerBindingPublication.swift` | `@MainActor` helper | Defers representable binding writes, rejects stale generations, and clears only the controller identity published by that view lifecycle. |
| `PiPVideoView.swift` | SwiftUI representable | Owns `PiPVideoView(player, controller: $binding)` construction, player swaps, coordinator lifetime, and platform host attachment without embedding backend implementation details. |
| `PiPVideoView+iOSNative.swift` | iOS native drawable plumbing | Drawable proxy and layout, generation-gated off-main callbacks, libVLC media controls, and the backend that wraps libVLC's system PiP controller for state and control. |
| `PiPVideoView+MacPrivate.swift` | macOS-only private API plumbing | `MacNativePiPBackend` orchestrates `MacPrivatePiPPresenter` (dynamic loader for `PIPViewController` from `PIP.framework`), `MacPrivatePiPDelegate`, and `MacNativePiPMediaController`. All private-framework symbol references live in this one file for security/audit review. Gated by `PiPController.allowsPrivateMacOSAPI` SPI. |
| `PixelBufferRenderer.swift` | `class PixelBufferRenderer: Sendable` | Direct sample-buffer path: format → lock → unlock → display. `CVPixelBufferPool` → `CMSampleBuffer` → layer. |
---
## C Interop Layer
### CLibVLC Target
```
Sources/CLibVLC/
├── include/vlc/ # Full libVLC 4.0 C headers
│ ├── vlc.h # Main umbrella header
│ ├── libvlc.h # Instance, logging, dialogs
│ ├── libvlc_media.h # Media creation, parsing, metadata
│ ├── libvlc_media_player.h # Player, tracks, events
│ ├── libvlc_media_list.h # Playlist
│ ├── libvlc_media_discoverer.h # Network discovery
│ ├── libvlc_renderer_discoverer.h # Renderer discovery
│ ├── libvlc_picture.h # Thumbnail generation
│ └── libvlc_events.h # Event types
└── shim.c # C shim for va_list formatting
```
### Why a C Shim?
Swift cannot directly consume C variadic functions (`va_list`). The shim provides:
```c
// shim.c — formats va_list into a fixed buffer before calling Swift
void swiftvlc_log_set(libvlc_instance_t *instance, void *opaque,
swiftvlc_log_cb callback);
```
This allows the logging callback to receive a pre-formatted `const char *` instead of a `va_list`.
### Linked Frameworks & Libraries
The xcframework links against the system frameworks and libraries
libVLC needs for decoding, rendering, and platform services:
**Frameworks:** AudioToolbox, AudioUnit\*, AVFoundation, AVKit, CoreAudio, CoreFoundation, CoreGraphics, CoreImage, CoreMedia, CoreServices, CoreText, CoreVideo, Foundation, IOKit\*, IOSurface, OpenGL\*, OpenGLES\*, QuartzCore, Security, SystemConfiguration, VideoToolbox
**Libraries:** libbz2, libc++, libiconv, libresolv, libsqlite3, libxml2, libz
\*Platform-conditional: AudioUnit/IOKit/OpenGL are macOS-only; OpenGLES is iOS/tvOS/visionOS-only.
---
## Concurrency & Threading Model
### Isolation Strategy
```mermaid
flowchart LR
subgraph MainActor["@MainActor — mutable state, SwiftUI"]
MA["Player · MediaListPlayer · Equalizer
VideoAdjustments · Marquee · Logo
PiPController · SwiftUI Views"]
end
subgraph VLCThreads["libVLC internal threads"]
VLC["Event callbacks · Logging
Decode (vmem)"]
end
subgraph Sendable["Sendable — any isolation"]
S["VLCInstance · Media · MediaList
Track · Metadata · MediaStatistics
PixelBufferRenderer · VLCError · all enums"]
end
VLCThreads -->|"AsyncStream yield"| MainActor
Sendable -.->|"safe to pass"| MainActor
Sendable -.->|"safe to pass"| VLCThreads
```
**Rules:**
1. **`@MainActor` types** own mutable state that SwiftUI observes. All property access and mutation happens on the main actor.
2. **`Sendable` types** are either immutable value types or use internal synchronization (`Mutex`, libVLC's own locks).
3. **C callbacks** fire on libVLC's internal threads. They yield values into `AsyncStream` continuations (which are thread-safe) or hop to the main actor where UI-adjacent state must change.
4. **`nonisolated(unsafe)`** is used for `OpaquePointer` fields that are only valid during the object's lifetime and accessed on the correct actor.
### Capturing C Pointers in `@Sendable` Closures
`OpaquePointer` and `UnsafeMutableRawPointer` aren't `Sendable` under Swift's region-based isolation, so getting them into `@Sendable` closures (`withTaskCancellationHandler` `onCancel`, `DispatchQueue.*.async`) needs one of two mechanisms:
- **`nonisolated(unsafe) let` local binding**, for captures into a single closure. The local opts out of isolation checking; the pointer is trivially transferable and stays valid for the enclosing scope. Used in `Media.parse`, `Player.deinit`, `MediaListPlayer.deinit`, `PixelBufferRenderer`, and the deinits in `DialogHandler`, `RendererDiscoverer`, and `MediaDiscoverer`.
```swift
nonisolated(unsafe) let p = pointer
DispatchQueue.global(qos: .utility).async {
libvlc_media_player_release(p)
}
```
- **`Mutex` with `State: @unchecked Sendable`**, for persistent storage of pointers that must be read or written from multiple threads. `Mutex`'s `sending` semantics require `State` to be sendable to the callee; marking `State: @unchecked Sendable` honors that while the `Mutex` itself provides the actual mutual exclusion. Used throughout the library — log-bridge installation state, thumbnail-request coordination, pixel-buffer renderer state, dialog-handler registration, broadcaster subscriber tables.
```swift
private struct State: @unchecked Sendable {
var selfBox: UnsafeMutableRawPointer?
}
private let state = Mutex(State())
```
Never use `Int(bitPattern:)` round-trips to launder pointers through `Sendable`. It loses type information and obscures intent.
---
## Event System
Three-layer architecture bridging C callbacks to SwiftUI:
```mermaid
flowchart TB
subgraph L1["C Callbacks — libVLC thread"]
CB["Player event types attached via libvlc_event_attach
playerEventCallback → mapEvent()"]
end
subgraph L2["AsyncStream Broadcasting — any thread"]
Store["EventBridge · Broadcaster
Each makeStream() creates an independent consumer"]
end
subgraph L3["Observable Properties — @MainActor"]
Update["Player event consumer Task
for await event → update @Observable props"]
end
CB -->|"yield to all continuations"| Store
Store -->|"consumed by"| Update
```
### PlayerEvent Cases
libVLC's player event types are attached to the event manager and mapped to typed Swift cases:
| Category | Swift Cases |
|---|---|
| **State** | `stateChanged(PlayerState)`, `mediaStopping`, `encounteredError` |
| **Time** | `timeChanged(Duration)`, `positionChanged(Double)`, `lengthChanged(Duration)` |
| **Capability** | `seekableChanged(Bool)`, `pausableChanged(Bool)` |
| **Tracks** | `tracksChanged`, `mediaChanged` |
| **Buffering** | `bufferingProgress(Float)` |
| **Audio** | `volumeChanged(Float)`, `muted`, `unmuted`, `corked`, `uncorked`, `audioDeviceChanged(String?)` |
| **Video** | `voutChanged(Int)`, `snapshotTaken(String)` |
| **Chapters** | `chapterChanged(Int)`, `titleListChanged`, `titleSelectionChanged(Int)` |
| **Recording** | `recordingChanged(isRecording:filePath:)` |
| **Programs** | `programAdded(Int)`, `programDeleted(Int)`, `programSelected(unselectedId:selectedId:)`, `programUpdated(Int)` |
### Multi-Consumer Broadcasting
`Broadcaster` (in [`Sources/SwiftVLC/Core/Broadcaster.swift`](Sources/SwiftVLC/Core/Broadcaster.swift)) consolidates this pattern for player events, log entries, dialog callbacks, renderer discovery events, and playback intent. Per-subscriber state (continuation, optional filter, lifecycle phase) lives under a single `Mutex`, so registration is one lock acquisition. `broadcast` snapshots the matching subscribers under the lock and yields outside it. Yielding resumes a consumer task and acquires its status-record lock; a concurrent task cancellation holds that same lock and calls `onTermination → unsubscribe → acquire Mutex`, so yielding while holding the `Mutex` would produce an AB-BA deadlock.
```swift
final class Broadcaster: Sendable {
func subscribe(bufferSize: Int? = nil, filter: Filter? = nil) -> AsyncStream
func broadcast(_ element: Element)
func finishAll() // closes current subscribers; allows resubscribe
func terminate() // closes current + auto-finishes future subscribes
}
```
The `terminate()` / `finishAll()` distinction matters because some broadcasters expose `subscribe` indirectly via a *computed* property — e.g. `DialogHandler.dialogs` returns `broadcaster.subscribe()` per access. After a producer is permanently gone (handler whose registration was rejected, discoverer whose pointer is being released), `terminate()` ensures every future `subscribe` call returns an immediately-finished stream so consumers don't hang. `finishAll()` is reserved for lazy producers with `onFirstSubscriber` reconciliation that may re-attach later.
`onFirstSubscriber` and `onLastUnsubscribed` callbacks let lazy producers attach to and detach from their upstream source only when there is actual demand. Both run on a per-broadcaster serial reconciliation `DispatchQueue` so concurrent subscribe/unsubscribe storms can't double-fire either side. The reconciliation is state-machine driven (`idle → scheduledOn → running → idle` and the symmetric off cycle), so a fast unsubscribe after a subscribe still terminates with the right callback.
### Lifecycle
1. **`Player.init()`** creates the `EventBridge` (which owns a `Broadcaster`) and attaches every event type to the libVLC event manager.
2. **`startEventConsumer()`** in `Player+Events.swift` spawns a `Task` that reads the bridge stream and updates the player's `@Observable` properties.
3. **`Player.deinit`** cancels the consumer task, then calls `EventBridge.invalidate()` to detach the C callbacks, finish the continuations, and release the store.
---
## Memory Management
### OpaquePointer Lifecycle
Every libVLC object follows the same pattern: **`init` allocates → use passes pointer → `deinit` releases**. Swift object lifetime owns C pointer lifetime.
| Swift Type | C Pointer | Alloc | Free |
|---|---|---|---|
| `VLCInstance` | `libvlc_instance_t*` | `libvlc_new` | `libvlc_release` |
| `Player` | `libvlc_media_player_t*` | `libvlc_media_player_new` | `libvlc_media_player_release` |
| `Media` | `libvlc_media_t*` | `libvlc_media_new_*` | `libvlc_media_release` |
| `MediaList` | `libvlc_media_list_t*` | `libvlc_media_list_new` | `libvlc_media_list_release` |
| `MediaListPlayer` | `libvlc_media_list_player_t*` | `libvlc_media_list_player_new` | `libvlc_media_list_player_release` |
| `MediaDiscoverer` | `libvlc_media_discoverer_t*` | `libvlc_media_discoverer_new` | `libvlc_media_discoverer_release` |
| `RendererDiscoverer` | `libvlc_renderer_discoverer_t*` | `libvlc_renderer_discoverer_new` | `libvlc_renderer_discoverer_release` |
| `RendererItem` | `libvlc_renderer_item_t*` | `libvlc_renderer_item_hold` | `libvlc_renderer_item_release` |
| `Equalizer` | `libvlc_equalizer_t*` | `libvlc_audio_equalizer_new` | `libvlc_audio_equalizer_release` |
### Unmanaged Patterns
For C callback contexts that need to bridge to Swift objects:
| Pattern | Use Case | Lifetime |
|---|---|---|
| `Unmanaged.passRetained` | Long-lived callback context or broadcaster box (EventBridge, logging, dialogs, renderer discovery) | Explicitly released in cleanup/deinit |
| `Unmanaged.passUnretained` | Short-lived reference (VideoSurface in `set_nsobject`) | Object must outlive the call |
### Deinit Ordering
Ordering in `Player.deinit` is load-bearing: detaching the listeners **before** releasing the player prevents use-after-free when a callback fires during teardown.
1. Cancel event consumer task
2. `EventBridge.invalidate()`
- Detach the C event listeners
- Finish all `AsyncStream` continuations
- Release retained store
3. `libvlc_media_player_stop_async()`
4. `libvlc_media_player_release()`
---
## Video Rendering
### VideoView Architecture
```mermaid
sequenceDiagram
participant SwiftUI
participant VideoView
participant VideoSurface as VideoSurface
(UIView / NSView)
participant libVLC
SwiftUI->>VideoView: makeUIView / makeNSView
VideoView->>VideoSurface: create
VideoSurface->>libVLC: set_nsobject(view pointer)
libVLC->>VideoSurface: adds rendering sublayer
loop On resize
VideoSurface->>VideoSurface: layoutSubviews() — sync sublayer frames
end
SwiftUI->>VideoView: dismantleUIView
VideoSurface->>libVLC: set_nsobject(nil)
```
There is no `CALayer` setup, no `MTKView`, and no `AVPlayerLayer` to configure. libVLC handles all rendering internally.
---
## Picture-in-Picture
PiP rendering depends on the platform:
| Path | Pipeline |
|---|---|
| **VideoView** | `set_nsobject` → VLC renders directly into the view |
| **PiPVideoView on iOS** | `set_nsobject` drawable proxy → libVLC's iOS sample-buffer video output → libVLC-owned `AVPictureInPictureController` |
| **Direct PiPController** | vmem callbacks → `CVPixelBuffer` → `CMSampleBuffer` → SwiftVLC-owned `AVSampleBufferDisplayLayer` → AVKit PiP |
| **PiPVideoView on macOS** | SPI opt-in: `set_nsobject` drawable → same VLC-owned `NSView` is moved into the system PiP presenter |
### iOS Native-drawable Path
`PiPVideoView` attaches a drawable proxy to the player. The proxy hosts
libVLC's inline rendering children and implements the Picture in Picture
selectors expected by VLC's bundled iOS video output. That output creates
and owns the `AVPictureInPictureController`, then passes its native window
controller back to SwiftVLC for start/stop, possibility, activity, playback
invalidation, and linear-playback policy. The view never hosts
`PiPController.layer`; that layer belongs exclusively to the direct path.
### Direct vmem Callback Pipeline
```mermaid
flowchart TB
subgraph PBR["PixelBufferRenderer — Mutex<State>"]
direction TB
F["1. formatCallback
Set BGRA format, create CVPixelBufferPool"]
L["2. lockCallback
Get buffer from pool, lock base address"]
U["3. unlockCallback
Unlock base address"]
D["4. displayCallback
Wrap as CMSampleBuffer, enqueue to layer"]
C["5. cleanupCallback
Release pool"]
F --> L --> U --> D --> C
end
subgraph PiPStack["Display Stack"]
Layer["AVSampleBufferDisplayLayer"]
Controller["AVPictureInPictureController
(managed by PiPController)"]
Layer --> Controller
end
D -->|"enqueues on dedicated serial queue"| Layer
```
The direct renderer requests 8-bit BGRA frames. Its optional resize pass
uses a device-RGB Core Image target, so this path is SDR and does not
preserve HDR or wide-color metadata.
### Direct Callback Ownership
libVLC copies vmem function pointers and their opaque value when a video
output opens. Clearing the media-player callback variables therefore does
not revoke a copy already held by that output. SwiftVLC ties each retained
opaque to one exact `libvlc_media_player_t` through
`NativePlayerHandleLifetime`:
- Sequential direct controllers on the same native handle reuse one stable
handle slot. During format negotiation, each vout replaces its copied
handle opaque with a separately retained vout context and decode pool. An
overlapping output therefore cannot change another output's dimensions,
pitch, pool, pending picture, or cleanup state. Each vout still resolves
its display target through the handle slot, so a successor controller can
atomically take over an already-open output.
- Replacing the native media-player handle creates a fresh slot for the new
handle and permanently retires the old slot.
- Every successful lock pins exactly one buffer until matching display, a
superseding successful lock, or that vout's cleanup. This balances the
retain even when pinned vmem suppresses display after native picture
construction fails. Cleanup drains only its own vout and releases that
vout context exactly once.
- Each `MediaListPlayer` native binding is a counted owner because pinned
libVLC retains the same `libvlc_media_player_t`. A rebuild synchronously
rebinds the retiring list player to an independent neutral player before
offloading its release, so it cannot advance or stop a successor's shared
handle. Final shutdown waits for the initial owner and every counted list
owner to release.
- Retirement suppresses new display work, but releases the handle opaque
only after the final counted native release returns for that exact handle
and every handle callback already in flight has drained. No timeout or
transient `vout` observation is treated as proof of safety.
This per-handle rule prevents both use-after-free during teardown and stale
controller teardown from clearing a successor's callbacks.
### Playback Range and Invalidation Policy
The direct AVKit playback delegate synchronously snapshots the current
native handle and maps media state as follows:
| Native state | AVKit range |
|---|---|
| No media | Invalid |
| Loaded media with length `<= 0` | Start zero, positive-infinite duration |
| Loaded media with positive length | Start zero, finite native duration |
An unknown duration is therefore valid live/indefinite content; rendering
does not wait for a finite length. Media, length, and seekability events
drive invalidation from their payloads. That is load-bearing because
`Player` and `PiPController` subscribe independently and either consumer
may observe the event first. Playback-state changes conservatively
invalidate as a payload-free fallback. Seekability payloads also synchronize
`requiresLinearPlayback` on the direct controller and iOS native backend.
### PiPController Responsibilities
1. **Timebase sync.** Creates a `CMTimebase` and keeps it aligned with the player's state (playing, paused, or rate-shifted).
2. **Playback delegation.** Owns a `PiPPlaybackDelegateProxy` that implements `AVPictureInPictureSampleBufferPlaybackDelegate`, breaking AVKit's strong-retain cycle while routing PiP play, pause, and seek commands into the `Player`.
3. **Range policy.** Reports no-content, live/indefinite, and finite media distinctly, and invalidates from event payloads rather than stale observable mirrors.
4. **State observation.** Distinguishes VLC-initiated state changes from pending PiP-initiated play/pause intent and keeps AVKit's linear-playback policy aligned with seekability.
5. **Deferred pause.** Debounces transient AVKit pause signals so skips and PiP transitions do not issue an unsafe short-lived native pause.
6. **Audio policy.** Configures the iOS playback category at construction but defers `setActive(true)` until PiP start or active playback intent. Direct and inactive native construction do not take audio focus; native adoption of an already-active Player activates before publishing backend ownership so automatic PiP cannot outrun session setup. A failed attempt remains retryable at observed native did-start.
On macOS, the SPI native backend intentionally bypasses SwiftVLC's vmem
renderer. It gives libVLC a normal `NSView` drawable and deliberately does
not conform that view to VLC's sample-buffer PiP protocols. When the
`PrivateMacOSPiP` SPI opt-in is enabled, entering PiP loads macOS's PiP
presenter at runtime and reparents the exact VLC drawable view into the
floating PiP controller. That keeps video, audio, play/pause intent, and
time on one VLC timeline while avoiding the macOS AVKit sample-buffer
`CALayerHost` mirror that can crop at 1:1 layer size instead of scaling
into the PiP panel.
### Mutually Exclusive
Only one rendering path may own a player at a time. On iOS,
`PiPVideoView` and `VideoView` both use `set_nsobject`, while a direct
`PiPController` replaces drawable rendering with vmem callbacks. A stable
same-handle callback slot makes sequential direct-controller handoff safe;
it does not make simultaneous drawable and vmem outputs supported. On
macOS, both SwiftUI views use `set_nsobject`, and the most recently attached
drawable owns VLC's native video output.
### Physical-device Validation
The iOS native backend deliberately reports PiP unavailable in Simulator.
Simulator AVKit can report an active sample-buffer PiP controller while the
system window remains black, so state-only simulator tests cannot validate
video delivery. End-to-end PiP rendering must be exercised on a physical
iOS device with the `audio` background mode enabled.
---
## Error Handling
### Typed Library Errors
Operations that introduce a SwiftVLC failure use `throws(VLCError)`:
```swift
func play() throws(VLCError) {
guard libvlc_media_player_play(pointer) == 0 else {
throw .playbackFailed(reason: "libVLC refused to start playback")
}
}
func parse(timeout: Duration) async throws(VLCError) -> Metadata {
// ...
throw .parseTimeout
}
```
Scoped closure APIs such as `withAdjustments`, `withMarquee`, `withLogo`,
and `MediaList.withLocked` use `rethrows`. They introduce no library
failure and propagate an error thrown by the caller's closure unchanged.
### Error Cases
| Error | When |
|---|---|
| `instanceCreationFailed` | `libvlc_new` returns nil |
| `mediaCreationFailed` | `libvlc_media_new_*` returns nil |
| `playbackFailed` | `libvlc_media_player_play` returns non-zero |
| `parseFailed` | Media parsing reports failure status |
| `parseTimeout` | Parsing exceeds specified timeout |
| `trackNotFound` | Track selection fails (invalid track ID) |
| `invalidState` | Operation attempted in wrong state |
| `invalidInput` | Public API argument is outside its documented range |
| `operationFailed` | Generic libVLC operation failure |
All errors conform to `LocalizedError` and `CustomStringConvertible` for logging and user-facing messages.
---
## Testing Strategy
### Overview
A comprehensive **Swift Testing** suite targets the public API surface.
Package tests are integration tests against the real libVLC binary.
Showcase UI tests use XCTest separately.
```
Tests/SwiftVLCTests/
├── Support/
│ ├── TestMedia.swift # Fixture URLs (bundled resources)
│ └── Tag.swift # Test tag definitions
├── Fixtures/
│ ├── test.mp4 # 1s, 64x64, with metadata
│ ├── twosec.mp4 # 2s, for seeking tests
│ ├── silence.wav # Audio-only
│ └── test.srt # Subtitle file
└── … # One test suite per domain area
```
### Test Tags
| Tag | Purpose | Speed |
|---|---|---|
| `logic` | Pure Swift logic, no libVLC | Fast |
| `integration` | Requires `VLCInstance` | Medium |
| `media` | Uses bundled fixture files | Medium |
| `mainActor` | Runs on `@MainActor` | Medium |
| `async` | Async tests with timeout guards | Slow |
### Testing Patterns
**Integration tests with real libVLC.** No mocking; every test creates actual `Player` and `Media` instances:
```swift
@Test(.tags(.integration, .media, .async))
func playAndWaitForState() async throws {
let player = Player()
try player.play(url: TestMedia.videoURL)
// Wait for state change...
}
```
**CI execution.** GitHub Actions runs package tests on `macos-latest`
with Xcode `latest-stable` plus the Swift 6.3.1 open-source toolchain
from swift.org, invoked via `xcrun --toolchain`. Showcase builds run on
`macos-26` with Xcode 26.4 because Xcode's built-in SwiftPM must parse
the Swift 6.3 manifest. The package test step is wrapped by
`scripts/ci-run-with-timeouts.py`, which enforces a 10-minute wall
clock and a 3-minute idle watchdog and sends SIGKILL to the process
group when either fires. Caches cover the libvlc xcframework, compiled
build products, and SPM dependency checkouts.
---
## Build & Release Infrastructure
### Scripts
| Script | Purpose |
|---|---|
| `scripts/setup-dev.sh` | First step for local repo work. Downloads the last-released xcframework into `Vendor/`, flips `Package.swift` to the local-path form, and points the Showcase app at the repo-local Swift package. Flags: `--force` (re-download), `--skip-download` (only flip local references). |
| `scripts/build-libvlc.sh` | Compiles libVLC from VideoLAN source (pinned via `VLC_HASH`) into `Vendor/libvlc.xcframework`. Applies the local VLC source patches described in README. |
| `scripts/fix-duplicate-symbols.sh` | Localizes `_json_parse_error` and `_json_read` in the chromecast plugin, which two VLC plugins each emit. Called automatically by `build-libvlc.sh` and `setup-dev.sh`. |
| `scripts/release.sh` | Cuts a versioned release, uploads the xcframework asset, pins the Showcase app to that exact Swift package version, and advances `main`. |
| `scripts/ci-use-released-xcframework.sh` | CI-only. Rewrites the current `Package.swift` `binaryTarget` to the url+checksum of the latest release tag. Run at CI job start so tests resolve against the same binary a downstream consumer would. |
| `scripts/ci-run-with-timeouts.py` | CI-only. Wraps `swift test` with wall-clock and idle timeouts; SIGKILLs the process group if either fires. |
### Package.swift resolution strategy
Published states (`main` and release tags) carry the remote form of the libvlc `binaryTarget`:
```swift
.binaryTarget(
name: "libvlc",
url: "https://github.com/harflabs/SwiftVLC/releases/download/vX.Y.Z/libvlc.xcframework.zip",
checksum: ""
)
```
That keeps the repository's default package state aligned with what downstream SPM consumers resolve. Local repo development flips the manifest back to the on-disk xcframework with `./scripts/setup-dev.sh`, which rewrites only the libvlc `binaryTarget` to:
```swift
.binaryTarget(name: "libvlc", path: "Vendor/libvlc.xcframework")
```
The Showcase apps follow the same split: published states pin `SwiftVLC` by exact release version, while `setup-dev.sh` rewrites the Xcode project to use the repo-local package checkout.
| Context | What `binaryTarget` looks like | Where the xcframework comes from |
|---|---|---|
| Published `main` | `url: + checksum:` of the latest release | GitHub Release asset |
| Local dev after `setup-dev.sh` | `path: "Vendor/libvlc.xcframework"` | `setup-dev.sh` (download) or `build-libvlc.sh` (build) |
| CI | rewritten in-memory to `url: + checksum:` of the latest release | SPM resolves + caches (keyed on checksum) |
| Release tag `vX.Y.Z` | `url: + checksum:` | `release.sh` uploads the zip as a release asset at that URL |
| SPM consumer pinning `X.Y.Z` | Reads the tag's `Package.swift` | SPM resolves + verifies checksum + caches |
### Release flow
`release.sh` creates a real release commit that stays on `main`.
```mermaid
flowchart LR
BUILD["build-libvlc.sh --all"] --> XCF["Vendor/libvlc.xcframework
(unstripped)"]
XCF --> RELEASE["release.sh X.Y.Z"]
RELEASE --> VERIFY["Verify all required slices present"]
VERIFY --> STRIP["strip -S"]
STRIP --> ZIP["ditto -c -k"]
ZIP --> SUM["swift package compute-checksum"]
SUM --> COMMIT["Commit on main:
Package.swift → url+checksum
Showcase → exactVersion X.Y.Z"]
COMMIT --> TAG["git tag vX.Y.Z"]
TAG --> PUSH_TAG["git push origin vX.Y.Z"]
PUSH_TAG --> GH["gh release create
+ attached .zip"]
GH --> PUSH_MAIN["git push origin HEAD:main"]
PUSH_MAIN --> SPM["main and consumers resolve the same release"]
```
Preflight refuses releases from non-`main` branches, uncommitted changes in `Package.swift` or the Showcase project, pre-existing local or remote tags, and unauthenticated `gh`. If a pre-commit rewrite or post-write sanity check fails, the script restores `Package.swift` and the Showcase project before exiting. The tag is pushed before `main`, so if GitHub Release creation fails, `origin/main` still points at the previous good release; finish the release or delete the tag before retrying. A post-write regex guard verifies that the rewritten `Package.swift` still contains the `CLibVLC` target, catching a malformed replacement before the tag is cut.
After publishing a tag, verify that Swift Package Index has completed its
asynchronous documentation build and that the unversioned documentation URL
resolves to the new release rather than the preceding tag.
### CI/CD
| Workflow | Trigger | Purpose |
|---|---|---|
| `test.yml` | Push to `main` / PR | Lints sources, builds all Showcase schemes, runs package tests with coverage, and checks public API doc coverage. |
| `sanitize.yml` | Push to `main`, selected PR paths, weekly schedule | Runs race, stress, memory, and lifecycle tests under Thread Sanitizer and Address Sanitizer. |
| `fixtures.yml` | Push to `main` / selected PR paths | Builds the layered dynamic-host fixtures and verifies that applications load exactly one copy of libVLC. |
| `vendor-manifest.yml` | Push to `main` / binary-manifest PR paths | Verifies every xcframework slice against its checked-in archive-member manifest. |
| `claude.yml` | Issue comment / PR mention | Claude Code bot integration. |
---
## Project Structure
```
SwiftVLC/
├── Sources/
│ ├── CLibVLC/ # C bridging layer
│ │ ├── include/vlc/ # libVLC 4.0 C headers
│ │ └── shim.c # va_list formatting shim
│ │
│ └── SwiftVLC/ # Main library
│ ├── Core/ # VLCInstance, VLCError, Logging, Duration
│ ├── Player/ # Player, EventBridge, PlayerState, Events, ABLoop, etc.
│ ├── Media/ # Media, Metadata, Track, Thumbnails, Statistics
│ ├── Audio/ # AudioOutput, Equalizer, ChannelModes
│ ├── Video/ # VideoView, AspectRatio, Adjustments, Marquee, Logo, Viewpoint
│ ├── Playlist/ # MediaList, MediaListPlayer, PlaybackMode
│ ├── Discovery/ # MediaDiscoverer, RendererDiscoverer
│ └── PiP/ # PiPController, PiPVideoView, PixelBufferRenderer
│
├── Tests/SwiftVLCTests/ # Swift Testing suite, one file per domain
│ ├── Support/ # TestMedia fixtures, Tag definitions
│ └── Fixtures/ # Bundled media files (~50 KB)
│
├── Showcase/ # Platform showcase apps
│ ├── Shared/ # Launch-arg/accessibility contracts, app icon, and shared showcase resources
│ ├── iOS/ # Full-featured iOS target/scheme, also enabled for Mac Catalyst
│ ├── macOS/ # Native macOS target/scheme with Mac-tailored showcases
│ ├── tvOS/ # Native tvOS target/scheme with TV-tailored showcases
│ ├── visionOS/ # Native visionOS target/scheme with focused playback coverage
│ └── UITests/
│ ├── iOS/ # UI tests for the iOS/Catalyst showcase
│ ├── macOS/ # Native macOS UI tests
│ └── tvOS/ # Empty tvOS UI-test target shell
│
├── Vendor/ # libvlc.xcframework (multi-GB unstripped; release zip a few hundred MB)
│
├── scripts/
│ ├── build-libvlc.sh # Compile libvlc from source
│ ├── setup-dev.sh # Download xcframework for local dev
│ ├── release.sh # Cut a versioned release and advance main
│ ├── fix-duplicate-symbols.sh # Localize duplicate json symbols
│ ├── ci-use-released-xcframework.sh # CI: point Package.swift at latest release
│ ├── ci-run-with-timeouts.py # CI: wall-clock + idle test timeouts
│ └── patches/ # Ordered VLC source patches applied by the build
│
├── .github/workflows/
│ ├── test.yml # CI test runner
│ ├── sanitize.yml # Sanitizer test runner
│ ├── fixtures.yml # Layered-consumer single-copy gate
│ ├── vendor-manifest.yml # XCFramework archive-member gate
│ └── claude.yml # Claude Code bot integration
│
├── Package.swift # SPM manifest (Swift 6.3+)
├── .swiftlint.yml # Lint configuration
├── .swiftformat # Format: 2-space indent
└── README.md # User guide
```