GPU Resources

The GPU resource endpoints return hosted and community capacity that can be used to create a GPU Node.

List available resources

curl -sS \
  -H "x-api-key: ${TEC_API_KEY}" \
  "${TEC_BASE_URL}/v1/projects/${TEC_PROJECT_ID}/gpu-resources"

Example response (resource values vary):

{
  "status": "success",
  "body": {
    "environment": "prod",
    "resources": [
      {
        "resource_id": "vm_gh8",
        "provider": "hosted",
        "region": "us-central1",
        "machine_type": "G-H100_80GB-x8",
        "display_name": "G-H100_80GB-x8",
        "gpu_model": "NVIDIA H100 80GB",
        "gpu_count_total": 8,
        "cpu_cores": 96,
        "memory_mb": 720000,
        "storage_gb": 1000,
        "gpu_count_available": 8,
        "price_per_hour_usd": 24
      },
      {
        "resource_id": "community_node-1",
        "provider": "community",
        "country": "US",
        "region": "CA",
        "gpu_model": "NVIDIA-RTX-4090",
        "gpu_memory_mb": 24576,
        "gpu_count_total": 1,
        "cpu_cores": 16,
        "memory_bytes": 68719476736,
        "storage_bytes": 1000000000000,
        "gpu_count_available": 1,
        "price_per_hour_usd": 0.13,
        "availability_percent": 99.5
      }
    ]
  }
}

Hosted capacity is grouped by region and machine type. Community availability excludes GPU capacity requested by GPU Node and other non-on-demand workloads. On-demand inference is preemptible and therefore does not reduce the reported rental availability.

Availability and pricing can change between discovery and creation. Read the latest resource response immediately before creating a deployment.

Get one resource

export TEC_RESOURCE_ID="community_node-1"

curl -sS \
  -H "x-api-key: ${TEC_API_KEY}" \
  "${TEC_BASE_URL}/v1/projects/${TEC_PROJECT_ID}/gpu-resources/${TEC_RESOURCE_ID}"

When the same hosted resource ID exists in more than one region, specify the region:

curl -sS \
  -H "x-api-key: ${TEC_API_KEY}" \
  "${TEC_BASE_URL}/v1/projects/${TEC_PROJECT_ID}/gpu-resources/${TEC_RESOURCE_ID}?region=us-central1"

List GPU Node templates

curl -sS \
  -H "x-api-key: ${TEC_API_KEY}" \
  "${TEC_BASE_URL}/v1/projects/${TEC_PROJECT_ID}/gpu-deployment-templates"

Creation accepts the GPU Node template and a container image returned in that template's container_images list. Use the returned template ID in the deployment request.