Truss Deployment Guide to Salad Portal and API

Introduction
Truss is a Python library that simplifies the deployment of machine learning models by creating containerized models that can be served via HTTP requests. This guide will show you how to deploy a Truss container to Salad Portal and Salad API so that your model can be accessed from anywhere.

Prerequisites
Before you begin, you will need the following:

Truss requires Python >=3.7 and <3.11. To install Truss from PyPi,
To install Truss, run this command: 

pip install truss

Here is a complete guide and docs to guide you through Truss

Step 1: Obtain Your API Key
To authenticate your API requests, you'll need an API key. To obtain your API key, follow these steps:

  • Log in to your Salad Cloud account at https://portal.salad.com
  • Click on your profile in the top right corner of the page, then click API Key.
  • Copy the API key that is displayed. You'll need this in the next step.

Step 2: Package Your Model
Before deploying your model to Salad Portal and API, you need to package it as a Truss container. Follow these steps:

  • Install the required dependencies for your model. For example, if you used scikit-learn to create your model, run: pip install --upgrade scikit-learn truss
  • Create your model using the Truss library. For example, if you used scikit-learn to create your model, run the following code: 
import truss
from sklearn.ensemble import RandomForestClassifier
from sklearn.datasets import load_iris

# Load the iris data set
iris = load_iris()
data_x = iris['data']
data_y = iris['target']

# Train the model
rfc = RandomForestClassifier()
rfc.fit(data_x, data_y)

# Create the Truss (serializing & packaging model)
tr = truss.create(rfc, target_directory="iris_rfc_truss")
  • Serve your model locally using Docker by running: 
truss run-image iris_rfc_truss.

The container will start running on port 8080.

  • Ensure that the container is running, then invoke the model as an API using a POST request:
curl -X POST http://127.0.0.1:8080/v1/models/model:predict -d '{"inputs": [[0, 0, 0, 0]]}'
  • Configure Your Model for Deployment :
    To configure your Truss, include a file config.yaml in the root directory of your Truss. Configuration is optional, as every configurable value has a sensible default. The Truss we generated in the quickstart sample provides a good example of a typical Truss config:
model_framework: sklearn
model_metadata:
  model_binary_dir: model
  supports_predict_proba: true
python_version: py39
requirements:
- scikit-learn==1.0.2
- threadpoolctl==3.0.0
- joblib==1.1.0
- numpy==1.20.3
- scipy==1.7.3

  • Create a Docker image for your Truss model by following the steps outlined in the Quickstart: making a Truss" section of the Truss documentation guide.
    Make sure to replace "iris_rfc_truss" with a name that reflects your model.

  • Upload your container to Dockerhub
    First, ensure that you have a DockerHub account. If you don't, create one at https://hub.docker.com/signup.

  • Next, create a Docker image for your Truss model. This can be done by running the following command in your terminal or command prompt:

  • Build the Docker image by running the following command in your terminal:

docker build -t <image-name> <path-to-Dockerfile>

  • Replace <image-name> with a name for your Docker image and <path-to-Dockerfile> with the path to the directory containing your Dockerfile.

  • Once the build process is complete, you can push the image to DockerHub by running the following command:

docker push <your_username>/<image_name>:<tag>
  • Test your Docker image by running the following command in your terminal: 
docker run -p 8080:8080 <your_username>/<image_name>:<tag>

  • This should start the Truss container and make it available at http://localhost:8080.

  • Stop the running Docker container by running the following command in your terminal: 

docker stop $(docker ps -q)

Step 3: Deploy the Container Image to Salad Portal.
Follow this step-by-step guide to deploy a container to Salad Portal here

Step 4: Deploy the Container Image to Salad API
To deploy your container image to Salad API, follow these steps:

  • Send a POST request to the following URL: 
https://api.salad.com/api/public/organizations/{organization_name}/projects/{project_name}/containers \

Replace organization_name} and {project_name} with the name of your organization and project, respectively.

  • In the headers of the POST request, include accept: application/json and content-type: application/json.
  • In the body of the POST request, include the following JSON data, replacing {API_KEY} with your Salad Cloud API key and {CONTAINER_IMAGE_URL} with the URL of your container image:
{
  "authorization_token": "{API_KEY}",
  "name": "{CONTAINER_NAME}",
  "image_url": "{CONTAINER_IMAGE_URL}"
}
  • Send a POST request using curl 
curl --request POST \
     --url https://api.salad.com/api/public/organizations/organization_name/projects/project_name/containers \
     --header 'accept: application/json' \
     --header 'content-type: application/json' \
     --data '
{
     "networking": {
          "protocol": "http"
     }
}
  • Check the status of the deployment by sending a GET request to the following URL: 
curl --request GET \
     --url https://api.salad.com/api/public/organizations/organization_name/projects/project_name/containers/container_group_name \
     --header 'accept: application/json'

Congratulations, You have successfully deployed a Truss container to Salad Portal and API! Remember to configure your Truss as needed for deployment and update it regularly. If you encounter any issues or have any questions, don't hesitate to reach out to the Salad support team.

Updated 6 months ago