Skip to main content
Ctrl+K

Flyte

  • User guide
  • Tutorials
  • Integrations
  • Deployment guide
  • API reference
    • Community
    • Flyte
  • GitHub
  • Slack
  • Flyte
  • User guide
  • Tutorials
  • Integrations
  • Deployment guide
  • API reference
  • Community
  • Flyte
  • GitHub
  • Slack
  • Flyte

API reference

  • flytekit
    • Overview
      • Model Files
      • Authoring Structure
      • Control Plane: FlyteRemote
      • Execution Time Support
      • Command Line Interfaces and Clients
    • Plugin API reference
      • AWS Athena
      • AWS Batch
      • Google Bigquery
      • Dask
      • Deck standard
        • flytekitplugins.deck.BoxRenderer
        • flytekitplugins.deck.FrameProfilingRenderer
        • flytekitplugins.deck.GanttChartRenderer
        • flytekitplugins.deck.ImageRenderer
        • flytekitplugins.deck.MarkdownRenderer
        • flytekitplugins.deck.SourceCodeRenderer
        • flytekitplugins.deck.TableRenderer
      • Dolt standard
      • Great expectations
      • Hive
      • Pod
      • KF MPI
      • KF Pytorch
        • flytekitplugins.kfpytorch.PyTorch
        • flytekitplugins.kfpytorch.Elastic
      • KF Tensorflow
      • Modin
      • Pandera
      • Papermill
      • PERIAN
      • Snowflake
      • Spark
      • SQLAlchemy
        • flytekitplugins.sqlalchemy.SQLAlchemyConfig
        • flytekitplugins.sqlalchemy.SQLAlchemyDefaultImages
        • flytekitplugins.sqlalchemy.SQLAlchemyTask
      • ONNX PyTorch
      • ONNX TensorFlow
      • ONNX ScikitLearn
      • Ray
      • DBT
      • Vaex
      • MLflow
      • DuckDB
      • SageMaker Inference
      • OpenAI
      • Inference
      • Neptune
      • Weights & Biases
    • Custom Tasks
      • Shell Task
      • SQLite3 Task
    • Custom Types
      • PyTorch Type
      • TensorFlow Type
      • Sklearn Type
    • Experimental Features
    • Pyflyte CLI
    • Flytekit Contribution Guide
  • flytectl
    • Overview
    • CLI Entrypoint
    • Verbs
      • flytectl create
      • flytectl completion
      • flytectl get
      • flytectl update
      • flytectl delete
      • flytectl register
      • flytectl config
      • flytectl compile
      • flytectl sandbox
      • flytectl demo
      • flytectl version
      • flytectl upgrade
    • Nouns
      • Project
        • flytectl create project
        • flytectl get project
        • flytectl update project
      • Execution
        • flytectl create execution
        • flytectl get execution
        • flytectl update execution
        • flytectl delete execution
      • Workflow
        • flytectl get workflow
        • flytectl update workflow-meta
      • Task
        • flytectl get task
        • flytectl update task-meta
      • Task resource attribute
        • flytectl get task-resource-attribute
        • flytectl update task-resource-attribute
        • flytectl delete task-resource-attribute
      • Cluster resource attribute
        • flytectl get cluster-resource-attribute
        • flytectl delete cluster-resource-attribute
        • flytectl update cluster-resource-attribute
      • Execution cluster label
        • flytectl get execution-cluster-label
        • flytectl update execution-cluster-label
        • flytectl delete execution-cluster-label
      • Execution queue attribute
        • flytectl get execution-queue-attribute
        • flytectl delete execution-queue-attribute
        • flytectl update execution-queue-attribute
      • Plugin override
        • flytectl get plugin-override
        • flytectl update plugin-override
        • flytectl delete plugin-override
      • Launchplan
        • flytectl get launchplan
        • flytectl update launchplan
        • flytectl update launchplan-meta
      • Workflow execution config
        • flytectl get workflow-execution-config
        • flytectl update workflow-execution-config
        • flytectl delete workflow-execution-config
      • Examples
        • flytectl register examples
      • Files
        • flytectl register files
      • Config
        • flytectl config validate
        • flytectl config init
        • flytectl config docs
        • flytectl config discover
      • Sandbox
        • flytectl sandbox start
        • flytectl sandbox status
        • flytectl sandbox teardown
        • flytectl sandbox exec
      • Demo
        • flytectl demo start
        • flytectl demo status
        • flytectl demo teardown
        • flytectl demo exec
        • flytectl demo reload
    • Contributing Guide
  • flyteidl
    • Flyte Admin Service entities
      • Protocol Documentation
    • Core Flyte language specification
      • Protocol Documentation
    • Flyte Data Catalog Service
      • Protocol Documentation
    • Flyte Internal and External Eventing interface
      • Protocol Documentation
    • Flyte Task Plugins
      • Protocol Documentation
    • REST and gRPC interface for the Flyte Admin Service
      • Protocol Documentation
    • Contributing to FlyteIDL
  • API reference
  • Flytekit API Reference
  • Plugin API reference
  • Model Inference reference

Model Inference reference#

Tags: Integration, Serving, Inference

previous

OpenAI reference

next

Neptune API reference
Edit on GitHub
Copyright © 2026, Flyte authors