Get started

Requirements Deploy CosmicAC Set up recommended model configurations

Install the CLI Create a GPU Container Job with the CLI Create a vLLM Managed Inference Job with the CLI Create a Parakeet Managed Inference Job with the CLI

Accounts and settings

Overview Teams and roles Manage teams Manage team members Manage nodes Accept a team invitation Settings pages and access

GPU Container Jobs

Overview Create a GPU Container Job in the web interface Connect to a GPU Container Job Set up SSH and Tailscale for a GPU Container Job

Managed Inference

Overview Replicas Model health

Check the health of a Managed Inference endpoint

Platform

Reference

Task deployment commands

Recommended configuration values Changelog

Get started

Deploy CosmicAC on your host machine, then create your first GPU Container Job or Managed Inference Job.

To start using CosmicAC, set up your deployment, install the CLI, and create your first job.

Set up CosmicAC

Deploy the CosmicAC stack on your host machine. After deployment, CosmicAC connects to your GPU Kubernetes cluster.

Complete these steps in order.

Prepare the cluster: confirm that the cluster meets the Kubernetes, GPU, virtualization, storage, and registry requirements. See Requirements.
Deploy the stack: deploy CosmicAC on your host machine with Docker Compose. See Deploy CosmicAC.
Set up recommended model configurations: add a recommended model configuration for each model you plan to serve. See Set up recommended model configurations.

Install the CLI

To create and manage jobs from your terminal, install the CosmicAC CLI.

Install the CLI

Add the CLI to your machine and set the URLs for your deployment.

Create your first job

Create a job with the CLI or in the web interface.

GPU Container Job

Create a GPU Container Job with the CLI

Run code on a GPU machine with shell access.

You can also create a GPU Container Job in the web interface.

Managed Inference Job

Create a vLLM Managed Inference Job with the CLI

Serve a language model behind an OpenAI-compatible API.

Create a Parakeet Managed Inference Job with the CLI

Serve a speech-to-text model behind an OpenAI-compatible API.

You can also create a vLLM Managed Inference Job or a Parakeet Managed Inference Job in the web interface.

Call a Managed Inference endpoint

Create an API key, then send requests to a vLLM endpoint or transcribe audio with a Parakeet endpoint.

Create an API key

Authorize requests to a Managed Inference endpoint with the CLI.

Call a vLLM endpoint

Send a chat completion request from the CLI or an OpenAI-compatible client.

Transcribe audio

Send audio to a Parakeet endpoint and get a transcription with the CLI or curl.

Welcome

Previous Page

Requirements

Next Page

On this page

Set up CosmicAC Install the CLI Create your first job GPU Container Job Managed Inference Job Call a Managed Inference endpoint