​

High Level

Regardless of your choice of stable diffusion inference server, models, or extensions, the basic process is as follows:

1. Get a docker image that runs your inference server
2. Copy any models and extensions you want into the docker image
3. Ensure the container is listening on an ipv6 address
4. Push the new image up to a container registry
5. Deploy the image as a SaladCloud container group

​

Find a Docker Image

Find a docker image of SD.Next. Here is one that we have verified works on Salad:

SD.Next - Git Repo: https://github.com/SaladTechnologies/sdnext - Docker Image: saladtechnologies/sdnext:base - Data Directory: /webui/data - Model Directory: /webui/data/models - Extension Directory: /webui/data/extensions - Controlnet Model Directory: /webui/extensions-builtin/sd-webui-controlnet/models

Note that you will be interacting with this as an API, and not through the browser user interface.

​

Download Your Models and Extensions

Download any model files you plan to use. For our example, we’re going to use Dreamshaper 8 , available on Civitai.com (https://blog.salad.com/civitai-salad/)

​

Create a Dockerfile

1. Create a new file called Dockerfile and open it in your preferred text editor. At this point, your directory should look like this:

   .
   ├── Dockerfile
   └── dreamshaper_8.safetensors
   

2. Copy the following into your Dockerfile:

# We're going to use this verified SD.Next image as a base
FROM saladtechnologies/sdnext:base

# Now we copy our model into the image
ENV MODEL_DIR=/webui/data/models
ENV CKPT_PATH=${MODEL_DIR}/Stable-diffusion/dreamshaper_8.safetensors
COPY dreamshaper_8.safetensors ${CKPT_PATH}

ENV PORT=7860
ENV HOST=[::]
# We pass in all of the relevant launch arguments to the SD.Next server via
# the CMD directive
ENTRYPOINT []
CMD ["/bin/bash", "-c", "${INSTALLDIR}/entrypoint.sh \
--data-dir=/webui/data \
--port ${PORT} \
--server-name ${HOST} \
--quick \
--use-cuda \
--docs \
--ckpt ${CKPT_PATH}"]

​

Build and Test Your Docker Image

1. Build the docker image. You should change the specified tag to suit your purpose.

docker build -t saladtechnologies/sdnext:dreamshaper-8 .

2. (Recommended) Run the docker image locally to confirm it works as expected

docker run -it --rm --gpus all -p 7860:7860 --name sdnext -e HOST=0.0.0.0 \
saladtechnologies/sdnext:dreamshaper-8

Navigate to http://localhost:7860/docs in your browser to see the API docs for SD.Next.

Note that we set the HOST environment variable to 0.0.0.0 for local development. The default value of HOST is [::], which is the ipv6 address that SaladCloud uses to route traffic to your container.

3. Test a Text-to-Image request

See the docs for more information on submitting a text-to-image request. Here’s an example JSON request body:

{
  "prompt": "A beautiful sunset over a calm lake"
}

Submit this to the /sdapi/v1/txt2img endpoint as a POST request:

curl -X 'POST' \
  'http://localhost:7860/sdapi/v1/txt2img' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
  "prompt": "A beautiful sunset over a calm lake"
}'

You will receive back a response in JSON format, including the generated image in Base64 encoded format:

{
  "images": ["base64encodedimage"],
  "parameters": {
    "prompt": "A beautiful sunset over a calm lake",
    "negative_prompt": "",
    "styles": null,
    "seed": -1,
    "subseed": -1,
    "subseed_strength": 0,
    "seed_resize_from_h": -1,
    "seed_resize_from_w": -1,
    "sampler_name": null,
    "batch_size": 1,
    "n_iter": 1,
    "steps": 50,
    "cfg_scale": 7,
    "width": 512,
    "height": 512,
    "restore_faces": null,
    "tiling": null,
    "do_not_save_samples": false,
    "do_not_save_grid": false,
    "eta": null,
    "denoising_strength": null,
    "s_min_uncond": null,
    "s_churn": null,
    "s_tmax": null,
    "s_tmin": null,
    "s_noise": null,
    "override_settings": null,
    "override_settings_restore_afterwards": true,
    "refiner_checkpoint": null,
    "refiner_switch_at": null,
    "disable_extra_networks": false,
    "comments": null,
    "enable_hr": false,
    "firstphase_width": 0,
    "firstphase_height": 0,
    "hr_scale": 2,
    "hr_upscaler": null,
    "hr_second_pass_steps": 0,
    "hr_resize_x": 0,
    "hr_resize_y": 0,
    "hr_checkpoint_name": null,
    "hr_sampler_name": null,
    "hr_prompt": "",
    "hr_negative_prompt": "",
    "sampler_index": "Euler",
    "script_name": null,
    "script_args": [],
    "send_images": true,
    "save_images": false,
    "alwayson_scripts": {}
  },
  "info": "{\"prompt\": \"A beautiful sunset over a calm lake\", \"all_prompts\": [\"A beautiful sunset over a calm lake\"], \"negative_prompt\": \"\", \"all_negative_prompts\": [\"\"], \"seed\": 726538242, \"all_seeds\": [726538242], \"subseed\": 4102306259, \"all_subseeds\": [4102306259], \"subseed_strength\": 0, \"width\": 512, \"height\": 512, \"sampler_name\": \"Euler\", \"cfg_scale\": 7.0, \"steps\": 50, \"batch_size\": 1, \"restore_faces\": false, \"face_restoration_model\": null, \"sd_model_name\": \"dreamshaper_8\", \"sd_model_hash\": \"879db523c3\", \"sd_vae_name\": null, \"sd_vae_hash\": null, \"seed_resize_from_w\": -1, \"seed_resize_from_h\": -1, \"denoising_strength\": null, \"extra_generation_params\": {}, \"index_of_first_image\": 0, \"infotexts\": [\"A beautiful sunset over a calm lake\\nSteps: 50, Sampler: Euler, CFG scale: 7.0, Seed: 726538242, Size: 512x512, Model hash: 879db523c3, Model: dreamshaper_8, Version: v1.7.0\"], \"styles\": [], \"job_timestamp\": \"20240731152922\", \"clip_skip\": 1, \"is_using_inpainting_conditioning\": false, \"version\": \"v1.7.0\"}"
}

4. Decode the base64 encoded string into your image. You can do this in a free browser tool such as https://codebeautify.org/base64-to-image-converter

or using CLI tools like jq and base64. For this method, first save your response to a file called response.json. Then, run the following command:

jq -r '.images[0]' response.json | base64 -d > image.png

​

Push and Deploy Your Docker Image

1. Push your docker image up to docker hub (or the container registry of your choice.)

docker push saladtechnologies/sdnext:dreamshaper-8

2. Deploy your image on Salad, using either the Portal or the SaladCloud Public API

   We’re going to name our container group something obvious, and fill in the configuration form. We’re going to use 3 replicas, to ensure coverage during node interruptions and reallocations.

   Since this is a stable diffusion 1.5 based model, we’re gong to give ourselves fairly modest hardware: 4 vCPUs, 12GB ram, and an RTX 3060 Ti GPU.

   We want to add the container gateway to our deployment, so that we will get a URL we can use to access it. Make sure to set the port to 7860, or whatever you set the PORT environment variable to in your Dockerfile.

   We need to enable a startup probe and a liveness probe, to make sure the container gateway only routes requests to nodes that are ready for them.

​

Interact with Your Deployment

1. Wait for the deployment to be ready.

   1. First, SaladCloud pulls your container image into our own internal high-performance cache.

   2. Next, SaladCloud locates eligible nodes for your workload, based on the configuration you provided.

   3. Next, SaladCloud begins downloading the cached container image to the nodes that have been assigned to your workload.

      This step can take tens of minutes in some cases, depending on the size of the image, and the internet speed of the individual nodes. Note that our progress bars are only estimates, and do not necessarily reflect real-time download status. These slow cold starts, and the possibility of nodes being interrupted by their host without warning, are why we always want to provision multiple replicas.

   4. Eventually, you will see instances listed as “running”, with a green check in the “ready” column.

2. Submit your prompt to the provided Access Domain Name. You will get back a json response within a few seconds. See above for how to submit the request and process the response.