inference-server

Pluggable Python HTTP web service (WSGI) for real-time AI/ML model inference compatible with Amazon SageMaker.

Implemented as a werkzeug WSGI application which can be served using a Gunicorn webserver, for example.

Contents

• Implementing server hooks
  • Hook definitions
  • Implementing model hooks
  • Implementing deserialization/serialization hooks
  • Registering the hooks
• Deployment
  • Docker container image
  • Configuring Gunicorn HTTP server
  • Configuring Gunicorn workers
• API Documentation
  • inference_server

Indices and tables

• Index

• Module Index

• Search Page