Skip to main content

Transcription

Transcription composes two owners. ctx.media acquires audio. ctx.ai selects an AI provider and turns that audio into text.

microphone · system audio · mixed audio · custom PCM stream
                         │
                         ▼
                 MediaAudioCaptureRef
                         │
                         ▼
            ctx.ai.transcription.stream()
                         │
             ┌───────────┴───────────┐
             ▼                       ▼
        fullStream                  final
      live typed parts       text · segments · warnings

Media capture does not select a transcription provider. Transcription does not open a microphone or screen behind an implicit option.

Capture and transcribe a microphone​

import {
  class AiTranscriptionErrorAiTranscriptionError,
  class MediaCaptureErrorMediaCaptureError,
  function command<TId extends string>(commandValue: CommandInput<TId>): Command<TId>
Declare an action the user runs from the command palette or a keybinding.

Reach for this when a person triggers the behavior. The `title` is what they
read in the palette, `category` groups the entry, and `when` decides whether
it shows up at all. The handler takes no argument, only a context.

Two neighbours are easy to confuse. `slashCommand` is also user-triggered,
but it is typed inside a chat session and receives free text plus the live
AI session. `tool` is not user-triggered at all, the model decides to call
it mid-turn.

The result goes in the mod's top-level `commands` array, alongside any slash
commands. The `id` is the address a keybinding or a `command:` hook targets,
so keep it stable.
@categoryAuthoring
command,
  function defineMod<const TMod extends ModDefinition>(definition: TMod & ValidateModHooks<TMod>): Mod
The entry point of the SDK. A mod's module default-exports one `defineMod`
call and nothing else.

The returned value is data. It registers nothing, starts nothing, and
touches no host state. The host folds it twice. At build time it runs the
module in a throwaway sandbox and reads the value to derive the serializable
manifest. At runtime it folds the same value again to wire your handlers
into their lanes: ext-host for commands, tools, effects, and hooks; the
sandboxed renderer for views and widgets.

That double fold is why the module must be pure. It is re-executed to
produce the manifest, so anything at the top level, a file read, a network
call, a timer, a `console` line, runs during a build with no host attached.
Put behavior inside handlers: `run`, `activate`, `handle`.

The `ValidateModHooks` constraint makes the call fail to typecheck when a
`hook.beforeAction` targets an action that is neither a known host action
nor `command:` plus one of this mod's own command ids.
@example```ts
export default defineMod({
  metadata: {
    displayName: 'Review',
    description: 'Review helpers for the current change.',
  },
  commands: [
    command({ id: 'review.open', title: 'Review: Open', run: openPanel }),
    slashCommand({ id: 'review', description: 'Review this change', run: review }),
  ],
  ai: {
    tools: [reviewTool],
  },
});
```@categoryAuthoring
defineMod,
} from '@modular/sdk';

const const transcribeMicrophone: Command<"transcription.microphone">transcribeMicrophone = command<"transcription.microphone">(commandValue: CommandInput<"transcription.microphone">): Command<"transcription.microphone">
Declare an action the user runs from the command palette or a keybinding.

Reach for this when a person triggers the behavior. The `title` is what they
read in the palette, `category` groups the entry, and `when` decides whether
it shows up at all. The handler takes no argument, only a context.

Two neighbours are easy to confuse. `slashCommand` is also user-triggered,
but it is typed inside a chat session and receives free text plus the live
AI session. `tool` is not user-triggered at all, the model decides to call
it mid-turn.

The result goes in the mod's top-level `commands` array, alongside any slash
commands. The `id` is the address a keybinding or a `command:` hook targets,
so keep it stable.
@categoryAuthoring
command({
  id: "transcription.microphone"id: 'transcription.microphone',
  title: stringtitle: 'Transcribe Microphone',
  run: (ctx: CommandContext) => Awaitable<unknown>run: async ctx: CommandContextctx => {
    const const capture: MediaCaptureOperationcapture = ctx: CommandContextctx.ContextBase.media: MediaApi
Screen, camera, and audio capture.
media.MediaApi.capture(request: MediaCaptureRequest): MediaCaptureOperationcapture({
      
audio: ({
    kind: "microphone";
    deviceId: string | null;
} | {
    kind: "system-audio";
})[]
audio: [{ kind: "microphone"kind: 'microphone', deviceId: string | nulldeviceId: null }],
      
video: {
    kind: "selected-source";
    source: Readonly<{
        id: string & $brand<"MediaSourceHandleId">;
        descriptor: Readonly<Readonly<{
            kind: "screen";
            sourceId: string;
            name: string;
        }>> | Readonly<Readonly<{
            kind: "window";
            sourceId: string;
            name: string;
        }>>;
    }>;
} | null
video: null,
    });
    const const mediaSession: MediaCaptureError | MediaCaptureSessionmediaSession = await const capture: MediaCaptureOperationcapture.MediaCaptureOperation.result: Promise<MediaCaptureError | MediaCaptureSession>result;
    if (const mediaSession: MediaCaptureError | MediaCaptureSessionmediaSession instanceof class MediaCaptureErrorMediaCaptureError) {
      ctx: CommandContextctx.ContextBase.logger: Logger
Your mod's log channel.
logger.Logger.error(message: string, ...args: readonly unknown[]): voiderror(const mediaSession: MediaCaptureErrormediaSession.Error.message: stringmessage);
      return;
    }

    const 
const audio: MediaCaptureError | Readonly<{
    kind: "capture-ref";
    id: string & $brand<"MediaAudioCaptureRefId">;
    format: Readonly<{
        encoding: "linear16";
        sampleRate: 16000;
        channels: 1;
    }>;
}>
audio = await const mediaSession: MediaCaptureSessionmediaSession.MediaCaptureSession.audio(): Promise<MediaAudioCaptureRef | MediaCaptureError>audio();
    if (
const audio: MediaCaptureError | Readonly<{
    kind: "capture-ref";
    id: string & $brand<"MediaAudioCaptureRefId">;
    format: Readonly<{
        encoding: "linear16";
        sampleRate: 16000;
        channels: 1;
    }>;
}>
audio instanceof class MediaCaptureErrorMediaCaptureError) {
      await const mediaSession: MediaCaptureSessionmediaSession.MediaCaptureSession.cancel(): Promise<void | MediaCaptureError>cancel();
      ctx: CommandContextctx.ContextBase.logger: Logger
Your mod's log channel.
logger.Logger.error(message: string, ...args: readonly unknown[]): voiderror(const audio: MediaCaptureErroraudio.Error.message: stringmessage);
      return;
    }

    const const result: AiStreamTranscriptionResultresult = ctx: CommandContextctx.ContextBase.ai: AiRuntimeApi
One off answers, agents, models, and sessions.
ai.AiRuntimeApi.transcription: AiTranscriptionApitranscription.AiTranscriptionApi.stream(request: AiStreamTranscriptionRequest): AiStreamTranscriptionResultstream({
      AiStreamTranscriptionRequest.model: AiTranscriptionModelSelectionmodel: { kind: "configured"kind: 'configured' },
      AiStreamTranscriptionRequest.audio: MediaAudioInputaudio,
      
AiStreamTranscriptionRequest.options?: Partial<{
    language: string | null;
    interimResults: boolean;
    timestamps: "none" | "segment" | "word";
    diarization: "disabled" | "preferred" | "required";
    punctuate: boolean;
    keywords: string[];
}> | undefined
options: {
        interimResults?: boolean | undefinedinterimResults: true,
        timestamps?: "none" | "segment" | "word" | undefinedtimestamps: 'word',
        diarization?: "disabled" | "preferred" | "required" | undefineddiarization: 'preferred',
        punctuate?: boolean | undefinedpunctuate: true,
      },
    });

    for await (const const part: AiTranscriptionStreamPartpart of const result: AiStreamTranscriptionResultresult.AiStreamTranscriptionResult.fullStream: AiAsyncIterableStream<AiTranscriptionStreamPart>fullStream) {
      switch (const part: AiTranscriptionStreamPartpart.type: "preparing" | "start" | "transcript-interim" | "transcript-segment" | "speech-start" | "speech-end" | "warning" | "error" | "finish"type) {
        case 'transcript-interim':
          ctx: CommandContextctx.ContextBase.logger: Logger
Your mod's log channel.
logger.Logger.debug(message: string, ...args: readonly unknown[]): voiddebug(
            
const part: {
    type: "transcript-interim";
    update: {
        kind: "append";
        delta: string;
    } | {
        kind: "replace";
        text: string;
    };
}
part.
update: {
    kind: "append";
    delta: string;
} | {
    kind: "replace";
    text: string;
}
update.kind: "append" | "replace"kind === 'append'
              ? 
const part: {
    type: "transcript-interim";
    update: {
        kind: "append";
        delta: string;
    } | {
        kind: "replace";
        text: string;
    };
}
part.
update: {
    kind: "append";
    delta: string;
}
update.delta: stringdelta
              : 
const part: {
    type: "transcript-interim";
    update: {
        kind: "append";
        delta: string;
    } | {
        kind: "replace";
        text: string;
    };
}
part.
update: {
    kind: "replace";
    text: string;
}
update.text: stringtext
          );
          break;
        case 'transcript-segment':
          if (
const part: {
    type: "transcript-segment";
    update: {
        kind: "upsert";
        segment: {
            id: string & $brand<"AiTranscriptionSegmentId">;
            text: string;
            speaker: {
                id: string & $brand<"AiTranscriptionSpeakerId">;
                label: string;
            } | null;
            startSecond: number;
            endSecond: number;
            words: {
                text: string;
                startSecond: number;
                endSecond: number;
                confidence: number | null;
            }[];
        };
    } | {
        kind: "remove";
        segmentId: string & $brand<"AiTranscriptionSegmentId">;
    };
}
part.
update: {
    kind: "upsert";
    segment: {
        id: string & $brand<"AiTranscriptionSegmentId">;
        text: string;
        speaker: {
            id: string & $brand<"AiTranscriptionSpeakerId">;
            label: string;
        } | null;
        startSecond: number;
        endSecond: number;
        words: {
            text: string;
            startSecond: number;
            endSecond: number;
            confidence: number | null;
        }[];
    };
} | {
    kind: "remove";
    segmentId: string & $brand<"AiTranscriptionSegmentId">;
}
update.kind: "upsert" | "remove"kind === 'upsert') {
            ctx: CommandContextctx.ContextBase.logger: Logger
Your mod's log channel.
logger.Logger.info(message: string, ...args: readonly unknown[]): voidinfo(
const part: {
    type: "transcript-segment";
    update: {
        kind: "upsert";
        segment: {
            id: string & $brand<"AiTranscriptionSegmentId">;
            text: string;
            speaker: {
                id: string & $brand<"AiTranscriptionSpeakerId">;
                label: string;
            } | null;
            startSecond: number;
            endSecond: number;
            words: {
                text: string;
                startSecond: number;
                endSecond: number;
                confidence: number | null;
            }[];
        };
    } | {
        kind: "remove";
        segmentId: string & $brand<"AiTranscriptionSegmentId">;
    };
}
part.
update: {
    kind: "upsert";
    segment: {
        id: string & $brand<"AiTranscriptionSegmentId">;
        text: string;
        speaker: {
            id: string & $brand<"AiTranscriptionSpeakerId">;
            label: string;
        } | null;
        startSecond: number;
        endSecond: number;
        words: {
            text: string;
            startSecond: number;
            endSecond: number;
            confidence: number | null;
        }[];
    };
}
update.
segment: {
    id: string & $brand<"AiTranscriptionSegmentId">;
    text: string;
    speaker: {
        id: string & $brand<"AiTranscriptionSpeakerId">;
        label: string;
    } | null;
    startSecond: number;
    endSecond: number;
    words: {
        text: string;
        startSecond: number;
        endSecond: number;
        confidence: number | null;
    }[];
}
segment.text: stringtext);
          }
          break;
        case 'warning':
          ctx: CommandContextctx.ContextBase.logger: Logger
Your mod's log channel.
logger.Logger.warn(message: string, ...args: readonly unknown[]): voidwarn(
const part: {
    type: "warning";
    warning: {
        code: "local-feature-limit" | "provider-warning";
        message: string;
    };
}
part.
warning: {
    code: "local-feature-limit" | "provider-warning";
    message: string;
}
warning.message: stringmessage);
          break;
        case 'error':
          ctx: CommandContextctx.ContextBase.logger: Logger
Your mod's log channel.
logger.Logger.error(message: string, ...args: readonly unknown[]): voiderror(
const part: {
    readonly type: "error";
    readonly error: AiTranscriptionError;
}
part.error: AiTranscriptionErrorerror.Error.message: stringmessage);
          break;
      }
    }

    const 
const final: AiTranscriptionError | {
    text: string;
    segments: {
        id: string & $brand<"AiTranscriptionSegmentId">;
        text: string;
        speaker: {
            id: string & $brand<"AiTranscriptionSpeakerId">;
            label: string;
        } | null;
        startSecond: number;
        endSecond: number;
        words: {
            text: string;
            startSecond: number;
            endSecond: number;
            confidence: number | null;
        }[];
    }[];
    warnings: {
        code: "local-feature-limit" | "provider-warning";
        message: string;
    }[];
    language: string | null;
    durationInSeconds: number | null;
}
final = await const result: AiStreamTranscriptionResultresult.
AiStreamTranscriptionResult.final: PromiseLike<AiTranscriptionError | {
    text: string;
    segments: {
        id: string & $brand<"AiTranscriptionSegmentId">;
        text: string;
        speaker: {
            id: string & $brand<"AiTranscriptionSpeakerId">;
            label: string;
        } | null;
        startSecond: number;
        endSecond: number;
        words: {
            text: string;
            startSecond: number;
            endSecond: number;
            confidence: number | null;
        }[];
    }[];
    warnings: {
        code: "local-feature-limit" | "provider-warning";
        message: string;
    }[];
    language: string | null;
    durationInSeconds: number | null;
}>
final;
    const const cleanup: void | MediaCaptureErrorcleanup = await const mediaSession: MediaCaptureSessionmediaSession.MediaCaptureSession.stop(): Promise<void | MediaCaptureError>stop();
    if (const cleanup: void | MediaCaptureErrorcleanup instanceof class MediaCaptureErrorMediaCaptureError) {
      ctx: CommandContextctx.ContextBase.logger: Logger
Your mod's log channel.
logger.Logger.error(message: string, ...args: readonly unknown[]): voiderror(const cleanup: MediaCaptureErrorcleanup.Error.message: stringmessage);
      return;
    }
    if (
const final: AiTranscriptionError | {
    text: string;
    segments: {
        id: string & $brand<"AiTranscriptionSegmentId">;
        text: string;
        speaker: {
            id: string & $brand<"AiTranscriptionSpeakerId">;
            label: string;
        } | null;
        startSecond: number;
        endSecond: number;
        words: {
            text: string;
            startSecond: number;
            endSecond: number;
            confidence: number | null;
        }[];
    }[];
    warnings: {
        code: "local-feature-limit" | "provider-warning";
        message: string;
    }[];
    language: string | null;
    durationInSeconds: number | null;
}
final instanceof class AiTranscriptionErrorAiTranscriptionError) {
      ctx: CommandContextctx.ContextBase.logger: Logger
Your mod's log channel.
logger.Logger.error(message: string, ...args: readonly unknown[]): voiderror(const final: AiTranscriptionErrorfinal.Error.message: stringmessage);
      return;
    }
    ctx: CommandContextctx.ContextBase.logger: Logger
Your mod's log channel.
logger.Logger.info(message: string, ...args: readonly unknown[]): voidinfo(
const final: {
    text: string;
    segments: {
        id: string & $brand<"AiTranscriptionSegmentId">;
        text: string;
        speaker: {
            id: string & $brand<"AiTranscriptionSpeakerId">;
            label: string;
        } | null;
        startSecond: number;
        endSecond: number;
        words: {
            text: string;
            startSecond: number;
            endSecond: number;
            confidence: number | null;
        }[];
    }[];
    warnings: {
        code: "local-feature-limit" | "provider-warning";
        message: string;
    }[];
    language: string | null;
    durationInSeconds: number | null;
}
final.text: stringtext);
  },
});

export default 
defineMod<{
    readonly metadata: {
        readonly displayName: "Transcription";
        readonly description: "Transcribe captured audio";
    };
    readonly commands: readonly [Command<"transcription.microphone">];
}>(definition: {
    readonly metadata: {
        readonly displayName: "Transcription";
        readonly description: "Transcribe captured audio";
    };
    readonly commands: readonly [Command<"transcription.microphone">];
}): Mod
The entry point of the SDK. A mod's module default-exports one `defineMod`
call and nothing else.

The returned value is data. It registers nothing, starts nothing, and
touches no host state. The host folds it twice. At build time it runs the
module in a throwaway sandbox and reads the value to derive the serializable
manifest. At runtime it folds the same value again to wire your handlers
into their lanes: ext-host for commands, tools, effects, and hooks; the
sandboxed renderer for views and widgets.

That double fold is why the module must be pure. It is re-executed to
produce the manifest, so anything at the top level, a file read, a network
call, a timer, a `console` line, runs during a build with no host attached.
Put behavior inside handlers: `run`, `activate`, `handle`.

The `ValidateModHooks` constraint makes the call fail to typecheck when a
`hook.beforeAction` targets an action that is neither a known host action
nor `command:` plus one of this mod's own command ids.
@example```ts
export default defineMod({
  metadata: {
    displayName: 'Review',
    description: 'Review helpers for the current change.',
  },
  commands: [
    command({ id: 'review.open', title: 'Review: Open', run: openPanel }),
    slashCommand({ id: 'review', description: 'Review this change', run: review }),
  ],
  ai: {
    tools: [reviewTool],
  },
});
```@categoryAuthoring
defineMod({
  
metadata: {
    readonly displayName: "Transcription";
    readonly description: "Transcribe captured audio";
}
metadata: {
    displayName: "Transcription"displayName: 'Transcription',
    description: "Transcribe captured audio"description: 'Transcribe captured audio',
  },
  commands: readonly [Command<"transcription.microphone">]commands: [const transcribeMicrophone: Command<"transcription.microphone">transcribeMicrophone],
});

The capture session remains the audio owner. Stop or cancel it after the transcription consumer finishes.

Provider catalog​

ctx.ai.transcription.providers.list() returns either provider entries or an AiTranscriptionError.

import { class AiTranscriptionErrorAiTranscriptionError, function command<TId extends string>(commandValue: CommandInput<TId>): Command<TId>
Declare an action the user runs from the command palette or a keybinding.

Reach for this when a person triggers the behavior. The `title` is what they
read in the palette, `category` groups the entry, and `when` decides whether
it shows up at all. The handler takes no argument, only a context.

Two neighbours are easy to confuse. `slashCommand` is also user-triggered,
but it is typed inside a chat session and receives free text plus the live
AI session. `tool` is not user-triggered at all, the model decides to call
it mid-turn.

The result goes in the mod's top-level `commands` array, alongside any slash
commands. The `id` is the address a keybinding or a `command:` hook targets,
so keep it stable.
@categoryAuthoring
command, function defineMod<const TMod extends ModDefinition>(definition: TMod & ValidateModHooks<TMod>): Mod
The entry point of the SDK. A mod's module default-exports one `defineMod`
call and nothing else.

The returned value is data. It registers nothing, starts nothing, and
touches no host state. The host folds it twice. At build time it runs the
module in a throwaway sandbox and reads the value to derive the serializable
manifest. At runtime it folds the same value again to wire your handlers
into their lanes: ext-host for commands, tools, effects, and hooks; the
sandboxed renderer for views and widgets.

That double fold is why the module must be pure. It is re-executed to
produce the manifest, so anything at the top level, a file read, a network
call, a timer, a `console` line, runs during a build with no host attached.
Put behavior inside handlers: `run`, `activate`, `handle`.

The `ValidateModHooks` constraint makes the call fail to typecheck when a
`hook.beforeAction` targets an action that is neither a known host action
nor `command:` plus one of this mod's own command ids.
@example```ts
export default defineMod({
  metadata: {
    displayName: 'Review',
    description: 'Review helpers for the current change.',
  },
  commands: [
    command({ id: 'review.open', title: 'Review: Open', run: openPanel }),
    slashCommand({ id: 'review', description: 'Review this change', run: review }),
  ],
  ai: {
    tools: [reviewTool],
  },
});
```@categoryAuthoring
defineMod } from '@modular/sdk';

export default 
defineMod<{
    readonly metadata: {
        readonly displayName: "Transcription Providers";
        readonly description: "Inspect transcription providers";
    };
    readonly commands: readonly [Command<string>];
}>(definition: {
    readonly metadata: {
        readonly displayName: "Transcription Providers";
        readonly description: "Inspect transcription providers";
    };
    readonly commands: readonly [Command<string>];
}): Mod
The entry point of the SDK. A mod's module default-exports one `defineMod`
call and nothing else.

The returned value is data. It registers nothing, starts nothing, and
touches no host state. The host folds it twice. At build time it runs the
module in a throwaway sandbox and reads the value to derive the serializable
manifest. At runtime it folds the same value again to wire your handlers
into their lanes: ext-host for commands, tools, effects, and hooks; the
sandboxed renderer for views and widgets.

That double fold is why the module must be pure. It is re-executed to
produce the manifest, so anything at the top level, a file read, a network
call, a timer, a `console` line, runs during a build with no host attached.
Put behavior inside handlers: `run`, `activate`, `handle`.

The `ValidateModHooks` constraint makes the call fail to typecheck when a
`hook.beforeAction` targets an action that is neither a known host action
nor `command:` plus one of this mod's own command ids.
@example```ts
export default defineMod({
  metadata: {
    displayName: 'Review',
    description: 'Review helpers for the current change.',
  },
  commands: [
    command({ id: 'review.open', title: 'Review: Open', run: openPanel }),
    slashCommand({ id: 'review', description: 'Review this change', run: review }),
  ],
  ai: {
    tools: [reviewTool],
  },
});
```@categoryAuthoring
defineMod({
  
metadata: {
    readonly displayName: "Transcription Providers";
    readonly description: "Inspect transcription providers";
}
metadata: {
    displayName: "Transcription Providers"displayName: 'Transcription Providers',
    description: "Inspect transcription providers"description: 'Inspect transcription providers',
  },
  commands: readonly [Command<string>]commands: [
    command<string>(commandValue: CommandInput<string>): Command<string>
Declare an action the user runs from the command palette or a keybinding.

Reach for this when a person triggers the behavior. The `title` is what they
read in the palette, `category` groups the entry, and `when` decides whether
it shows up at all. The handler takes no argument, only a context.

Two neighbours are easy to confuse. `slashCommand` is also user-triggered,
but it is typed inside a chat session and receives free text plus the live
AI session. `tool` is not user-triggered at all, the model decides to call
it mid-turn.

The result goes in the mod's top-level `commands` array, alongside any slash
commands. The `id` is the address a keybinding or a `command:` hook targets,
so keep it stable.
@categoryAuthoring
command({
      id: stringid: 'transcription.list-providers',
      title: stringtitle: 'List Transcription Providers',
      run: (ctx: CommandContext) => Awaitable<unknown>run: async ctx: CommandContextctx => {
        const 
const providers: AiTranscriptionError | readonly {
    descriptor: Readonly<{
        id: string & $brand<"AiTranscriptionProviderId">;
        name: string;
        description: string;
        processing: "local" | "remote";
        models: readonly Readonly<{
            id: string & $brand<"AiTranscriptionModelId">;
            name: string;
            mode: "streaming";
            features: readonly (Readonly<{
                kind: "interim";
                updates: "append" | "append-and-replace" | "replace";
            }> | Readonly<{
                kind: "timing";
                granularity: "segment" | "word";
            }> | Readonly<{
                kind: "speaker-diarization";
            }> | Readonly<{
                kind: "language-detection";
            }> | Readonly<...> | Readonly<...>)[];
        }>[];
    }>;
    status: {
        ...;
    } | ... 3 more ... | {
        ...;
    };
}[]
providers = await ctx: CommandContextctx.ContextBase.ai: AiRuntimeApi
One off answers, agents, models, and sessions.
ai.AiRuntimeApi.transcription: AiTranscriptionApitranscription.AiTranscriptionApi.providers: AiTranscriptionProvidersApiproviders.AiTranscriptionProvidersApi.list(): Promise<readonly AiTranscriptionProviderCatalogEntry[] | AiTranscriptionError>list();
        if (
const providers: AiTranscriptionError | readonly {
    descriptor: Readonly<{
        id: string & $brand<"AiTranscriptionProviderId">;
        name: string;
        description: string;
        processing: "local" | "remote";
        models: readonly Readonly<{
            id: string & $brand<"AiTranscriptionModelId">;
            name: string;
            mode: "streaming";
            features: readonly (Readonly<{
                kind: "interim";
                updates: "append" | "append-and-replace" | "replace";
            }> | Readonly<{
                kind: "timing";
                granularity: "segment" | "word";
            }> | Readonly<{
                kind: "speaker-diarization";
            }> | Readonly<{
                kind: "language-detection";
            }> | Readonly<...> | Readonly<...>)[];
        }>[];
    }>;
    status: {
        ...;
    } | ... 3 more ... | {
        ...;
    };
}[]
providers instanceof class AiTranscriptionErrorAiTranscriptionError) {
          ctx: CommandContextctx.ContextBase.logger: Logger
Your mod's log channel.
logger.Logger.error(message: string, ...args: readonly unknown[]): voiderror(const providers: AiTranscriptionErrorproviders.Error.message: stringmessage);
          return;
        }

        for (const 
const provider: {
    descriptor: Readonly<{
        id: string & $brand<"AiTranscriptionProviderId">;
        name: string;
        description: string;
        processing: "local" | "remote";
        models: readonly Readonly<{
            id: string & $brand<"AiTranscriptionModelId">;
            name: string;
            mode: "streaming";
            features: readonly (Readonly<{
                kind: "interim";
                updates: "append" | "append-and-replace" | "replace";
            }> | Readonly<{
                kind: "timing";
                granularity: "segment" | "word";
            }> | Readonly<{
                kind: "speaker-diarization";
            }> | Readonly<{
                kind: "language-detection";
            }> | Readonly<...> | Readonly<...>)[];
        }>[];
    }>;
    status: {
        ...;
    } | ... 3 more ... | {
        ...;
    };
}
provider of 
const providers: readonly {
    descriptor: Readonly<{
        id: string & $brand<"AiTranscriptionProviderId">;
        name: string;
        description: string;
        processing: "local" | "remote";
        models: readonly Readonly<{
            id: string & $brand<"AiTranscriptionModelId">;
            name: string;
            mode: "streaming";
            features: readonly (Readonly<{
                kind: "interim";
                updates: "append" | "append-and-replace" | "replace";
            }> | Readonly<{
                kind: "timing";
                granularity: "segment" | "word";
            }> | Readonly<{
                kind: "speaker-diarization";
            }> | Readonly<{
                kind: "language-detection";
            }> | Readonly<...> | Readonly<...>)[];
        }>[];
    }>;
    status: {
        ...;
    } | ... 3 more ... | {
        ...;
    };
}[]
providers) {
          ctx: CommandContextctx.ContextBase.logger: Logger
Your mod's log channel.
logger.Logger.info(message: string, ...args: readonly unknown[]): voidinfo(
            `${
const provider: {
    descriptor: Readonly<{
        id: string & $brand<"AiTranscriptionProviderId">;
        name: string;
        description: string;
        processing: "local" | "remote";
        models: readonly Readonly<{
            id: string & $brand<"AiTranscriptionModelId">;
            name: string;
            mode: "streaming";
            features: readonly (Readonly<{
                kind: "interim";
                updates: "append" | "append-and-replace" | "replace";
            }> | Readonly<{
                kind: "timing";
                granularity: "segment" | "word";
            }> | Readonly<{
                kind: "speaker-diarization";
            }> | Readonly<{
                kind: "language-detection";
            }> | Readonly<...> | Readonly<...>)[];
        }>[];
    }>;
    status: {
        ...;
    } | ... 3 more ... | {
        ...;
    };
}
provider.
descriptor: Readonly<{
    id: string & $brand<"AiTranscriptionProviderId">;
    name: string;
    description: string;
    processing: "local" | "remote";
    models: readonly Readonly<{
        id: string & $brand<"AiTranscriptionModelId">;
        name: string;
        mode: "streaming";
        features: readonly (Readonly<{
            kind: "interim";
            updates: "append" | "append-and-replace" | "replace";
        }> | Readonly<{
            kind: "timing";
            granularity: "segment" | "word";
        }> | Readonly<{
            kind: "speaker-diarization";
        }> | Readonly<{
            kind: "language-detection";
        }> | Readonly<...> | Readonly<...>)[];
    }>[];
}>
descriptor.name: stringname}: ${
const provider: {
    descriptor: Readonly<{
        id: string & $brand<"AiTranscriptionProviderId">;
        name: string;
        description: string;
        processing: "local" | "remote";
        models: readonly Readonly<{
            id: string & $brand<"AiTranscriptionModelId">;
            name: string;
            mode: "streaming";
            features: readonly (Readonly<{
                kind: "interim";
                updates: "append" | "append-and-replace" | "replace";
            }> | Readonly<{
                kind: "timing";
                granularity: "segment" | "word";
            }> | Readonly<{
                kind: "speaker-diarization";
            }> | Readonly<{
                kind: "language-detection";
            }> | Readonly<...> | Readonly<...>)[];
        }>[];
    }>;
    status: {
        ...;
    } | ... 3 more ... | {
        ...;
    };
}
provider.
status: {
    kind: "unavailable";
    reason: "disabled" | "not-configured" | "not-supported";
    message: string;
} | {
    kind: "preparing";
    stage: "connecting" | "loading-model";
} | {
    kind: "ready";
} | {
    kind: "busy";
    message: string;
} | {
    kind: "failed";
    message: string;
}
status.kind: "unavailable" | "preparing" | "ready" | "busy" | "failed"kind}`
          );
        }
      },
    }),
  ],
});

Each catalog entry keeps an immutable descriptor beside current status. The descriptor owns provider identity, processing location, and model features. Status owns availability, preparation, readiness, busy state, or failure.

Model selection​

The request uses one closed union:

SelectionMeaning
{ kind: 'configured' }Use the application's configured provider and model
{ kind: 'model', providerId, modelId }Address one catalog entry explicitly

Explicit selection does not change application settings.

Stream parts​

fullStream yields one ordered discriminated union:

PartMeaning
preparingResolving a provider, loading a model, or connecting
startResolved route and local or remote processing location
transcript-interimAppend or replace unstable text
transcript-segmentUpsert or remove a stable identified segment
speech-start, speech-endVoice activity boundaries
warningRecoverable provider or capability warning
errorTyped terminal AiTranscriptionError
finishStopped, input closed, provider ended, or cancelled

Segment identity is stable across revisions. Update presentation by segment.id, not by array position and not by appending every revision.

One consumption lane​

Choose live streaming or terminal-only consumption:

First claimBehavior
Iterate fullStreamThe caller owns the one live iterator; terminal promises settle from it
Await final, text, or segmentsThe result drains internally and exposes terminal projections

Do not create a second iterator or claim fullStream after terminal-only consumption begins. Those are invariant errors.

stop() asks the provider to finish normally. cancel(reason?) aborts input, provider work, and owned transport. Failures are AiTranscriptionError values with a stable code, operation, provider identity, and recoverability flag.