sdnext-dynamic

This wraps our SDNext image with an entrypoint script that downloads a checkpoint at runtime based on an environment variable.

You can use the prebuilt docker image at:

saladtechnologies/sdnext:dynamic

The Github Repo is available here: https://github.com/SaladTechnologies/sdnext-dynamic

"Baked" Images

If you want an image with the model preloaded, you can use one of the "baked" images. These are built with the build-baked script, and will not need to download the models at runtime.

• saladtechnologies/sdnext:sdxl1.0 - Preloaded with SDXL Base and Refiner
• saladtechnologies/sdnext:dreamshaper8 - Preloaded with DreamShaper 8
• saladtechnologies/sdnext:rundiffusionxl-beta - Preloaded with RunDiffusion XL Beta and SDXL Refiner
• saladtechnologies/sdnext:dreamshaper8-qr-code - Preloaded with DreamShaper 8 and QR Code Monster

Build your own image

Without a model preloaded

The "baked" images use this one as a base

## Usage: ./build [--push]
./scripts/build

With a model preloaded

Usage: ./build-baked [OPTIONS]

Options:
  --push                          Push the built image to DockerHub. Default: false
  --civitai-version-ids <v1,v2>   Comma-separated list of CivitAI model version IDs to load. Default: ""
  --load-refiner                  Load the refiner model. Default: false
  --load-sdxl-base                Load the SDXL base model. Default: false
  --controlnet-urls <url1,url2>   Comma-separated list of ControlNet download URLs. Default: ""
  --extension-urls <url1,url2>    Comma-separated list of extension Git URLs. Default: ""
  --ckpt-urls <url1,url2>         Comma-separated list of CKPT download URLs. Default: ""
  --vae-urls <url1,url2>          Comma-separated list of VAE download URLs. Default: ""
  --lora-urls <url1,url2>         Comma-separated list of LoRa download URLs. Default: ""
  --tag <tag>                     Tag to use for the image. Defaults to the CivitAI version IDs with a hyphen between them
  --image <image>                 Image to build. Default: saladtechnologies/sdnext
  --help                          Show this message and exit

Examples

Load 2 models from civitai

./scripts/build-baked --civitai-version-ids 122143,128713

Load SDXL Base and Refiner, and push the image to your own repository with a custom tag

./scripts/build-baked --load-sdxl-base --load-refiner --push --tag sdxl1.0 --image my-org/my-repo

Original SDXL + Refiner

./scripts/build-sdxl

Environment Variables

Built-in Exec Probes

An exec probe is a command that is run periodically to check the health or readiness of the container. It returns a zero exit code if the container is healthy, and a non-zero exit code if the container is unhealthy. You can configure the maximum number of retries and the interval between retries using either the portal or the Create Container Group API. Learn more about health probes in the docs.

Startup Probe

This probe will return a zero exit code when the server is ready to accept connections. It will return a non-zero exit code if the server is not ready to accept connections. Keep in mind it can take quite a while to download the models, so be sure to set a generous number of retries and a long interval, or your container will never start taking traffic, and will be continually reallocated.

For example, SDXL takes quite a while to load the model, and one of the things the startup probe does is enable the refiner (if you've configured the container to do so). Enabling the refiner can take up to 3 minutes in rare circumstances, so the maximum timeout should be at least 180 seconds. The total time required to reach readiness could be quite long, so we want to make sure the failure threshold allows for enough time. In this case, I've set it to 600, which combined with the period, gives us 600 seconds (10 minutes) for the container to come up. For a container using an SD 1.5 model, you could get away with considerably shorter timeouts and failure thresholds. You will want to experiment with your specific model(s) to find the right values.

Portal

• Protocol: exec
• Command: Command: python , Arg: /probes/readiness.py
• Initial Delay Seconds: 20
• Period Seconds: 1
• Timeout Seconds: 200
• Success Threshold: 1
• Failure Threshold: 600

API

{
  "exec": {
    "command": [
      "python",
      "/probes/readiness.py"
    ]
  },
  "initial_delay_seconds": 20,
  "period_seconds": 1,
  "timeout_seconds": 200,
  "success_threshold": 1,
  "failure_threshold": 600
}

Liveness Probe

This probe will return a zero exit code when the server has not encountered any memory problems. It will return a non-zero exit code if the server has run out of, or is about to run out of, memory.

Since this probe primarily checks for memory issues, it should be configured with a short timeout and a low failure threshold. If the container is running out of memory, it will likely not be able to recover, so we want to fail fast and let the container be reallocated.

Portal

• Protocol: exec
• Command: Command: python , Arg: /probes/healthcheck.py
• Initial Delay Seconds: 600
• Period Seconds: 30
• Timeout Seconds: 2
• Success Threshold: 1
• Failure Threshold: 3

API

 {
  "exec": {
    "command": [
      "python",
      "/probes/healthcheck.py"
    ]
  },
  "initial_delay_seconds": 600,
  "period_seconds": 30,
  "timeout_seconds": 2,
  "success_threshold": 1,
  "failure_threshold": 3
}

Finding Your Model Version ID (Website)

1. Navigate to the Civit.ai page for the model you want to use

   

2. Click the version of the model you want to download

   

3. Copy the version ID from the URL

   

OR

2. Copy the link from the download button
   

3. Paste the link into a text editor and copy the model version ID from the URL

https://civitai.com/api/download/models/128713

For this url, the model version id is 128713.

Finding Your Model Version ID (API)

1. Navigate to the Civit.ai page for the model you want to use
   

2. Grab the model ID from the URL
   

3. Use the API to get the model versions

curl  -X GET \
  'https://civitai.com/api/v1/models/4384'

You'll get a JSON response that includes a .modelVersions field:

"modelVersions": [
    {
      "id": 128713,
      "modelId": 4384,
      "name": "8",
      "createdAt": "2023-07-29T15:56:46.392Z",
      "updatedAt": "2023-08-01T13:38:21.400Z",
      "trainedWords": [],
      "trainingStatus": null,
      "trainingDetails": null,
      "baseModel": "SD 1.5",
      "baseModelType": "Standard",
      "earlyAccessTimeFrame": 0,
      "description": "<ul><li><p>Better at handling Character LoRA</p></li><li><p>Better at photorealism without sacrificing range (can still do art and anime pretty well, even if you might have to increase weight)</p></li><li><p>Better at NSFW</p></li></ul>",
      "vaeId": null,
      "stats": {
        "downloadCount": 157806,
        "ratingCount": 673,
        "rating": 4.9
      },
      "files": []
      ...
    }
]

Find the model version you want to use and copy the id field.

Deploying on Salad (Portal)

As an example, we're going to deploy the Dreamshaper XL model

Create a Deployment

1. Name your deployment
2. Select the saladtechnologies/sdnext:dynamic image
3. Select how many vCPUs you need. For this we'll use 2
4. Select how much RAM you need. Since we plan on using the refiner model, we'll use 30GB

5. Select your GPUs. For an SDXL based model, we'll want GPUs with at least 16GB of VRAM.

6. Click "edit" on the Container Gateway section, and enable networking for port 7860 (the default port for SDNext).

7. Click "edit" on the Environment Variables section, and add the following variables:
   • HOST with a value of [::]
   • CIVITAI_MODEL_VERSION_IDS with a value of 126688
   • LOAD_REFINER with a value of 1

8. Click "Deploy" to finish creating your deployment. Then, click "Start" to start it.

Your Finished Deployment

With the environment configured like this: