Text-to-Video

Base URLs:

• https://api.novita.ai

POST Text-to-Video

POST /v3/async/txt2video

This API generates new videos using text.

Request header parameters

• Content-TypestringRequired

  Only application/json supported

• AuthorizationstringRequired

  In Bearer {{Key}} format.

Request Body parameters

• extraobject

  Show properties

• model_namestringRequired

  Name of SD1.x checkpoints, you can call the /v3/models endpoint to retrieve the sd_name_in_api field as the model_name.

• heightintegerRequired

  height of video, range [256, 1024]

• widthintegerRequired

  width of video, range [256, 1024]

• seedintegerRequired

  seed, Minmum -1

• stepsintegerRequired

  steps, Range [1. 50]

• negative_promptstring

  Negtive prompt word, divided by `,`, Range [1, 1024]

• prompts[object]Required

  The total number of frames in prompts need less than or equal to 128, where the total number of frames is the cumulative sum of all prompt frames.

  Show properties

• guidance_scalenumber

  This setting says how close the server will listen to your prompt. Range: [1, 30]

• loras[object]

  Lora options, currenlty supports up to 5 Lora

  Show properties

• embeddings[object]

  Textual Inversion options, currenlty supports up to 5 embeddings

  Show properties

• closed_loopboolean

  closed_loop lets you control how the animation loops back to the first frame.

• clip_skipnumber

  This parameter indicates the number of layers to stop from the bottom during optimization, so clip_skip on 2 would mean, that in SD1.x model where the CLIP has 12 layers, you would stop at 10th layer, Range [1, 12]

Responses

• task_idstring

Example request

curl --location 'https://api.novita.ai/v3/async/txt2video' --header 'Content-Type: application/json' --header 'Authorization: Bearer {{key}}' --data '{ 'extra': { 'response_video_type': 'mp4' }, 'model_name': 'dreamshaper_8_93211.safetensors', 'height': 512, 'width': 768, 'seed': 20231219, 'steps': 20, 'negative_prompt': '(deformed iris, deformed pupils, semi-realistic, cgi, 3d, render, sketch, cartoon, drawing, anime, mutated hands and fingers:1.4), (deformed, distorted, disfigured:1.3), poorly drawn, bad anatomy, wrong anatomy, extra limb, missing limb, floating limbs, disconnected limbs, mutation, mutated, ugly, disgusting, amputation', 'prompts': [ { 'frames': 16, 'prompt': 'Portrait, 18 year old, (crowd groping: 1.7) Latina, woman, large breasts, skinny, abs, completely naked, legs spread open, vagina spread open, (crowd surfing at a octopus concert), laughing, pouring rain, soaking wet' } ], 'guidance_scale': 7.5, 'loras': [ { 'model_name': 'add_detail_44319', 'strength': 0.8 } ], 'embeddings': [ { 'model_name': '' } ], 'closed_loop': false, 'clip_skip': 3 }'

Response

200200

{ 'task_id': '0618e8fb-15ec-4e33-8aa0-253ddb4f07aa' }

Example

This API helps to generate video with text. The returned video can be access by API /v3/async/task-result with task_id.

Please set the Content-Type header to application/json in your HTTP request to indicate that you are sending JSON data. Currently, only JSON format is supported.

Try it in playground.

Request:

curl --location 'https://api.novita.ai/v3/async/txt2video' \
--header 'Authorization: Bearer {{key}}' \
--header 'Content-Type: application/json' \
--data '{
    "model_name": "darkSushiMixMix_225D_64380.safetensors",
    "height": 512,
    "width": 512,
    "steps": 20,
    "seed": -1,
    "prompts": [
        {
            "frames": 32,
            "prompt": "in the wintry dusk, a little girl, holds matches tightly"
        },
        {
            "frames": 32,
            "prompt": "a little girl, barefoot on the frosty pavement, she seeks solace."
        },
        {
            "frames": 32,
            "prompt": "a little girl, with each match, a fleeting dance of warmth and hope."
        },
        {
            "frames": 32,
            "prompt": "a little girl,in the quiet night, her silent story unfolds."
        }
    ],
    "negative_prompt": "nsfw,ng_deepnegative_v1_75t, badhandv4, (worst quality:2),(low quality:2), (normal quality:2), lowres,((monochrome)), ((grayscale)),watermark"
}'

Response:

{
    "task_id": "fa20dff3-18cb-4417-a7f8-269456a35154"
}

Use task_id to get images

HTTP status codes in the 2xx range indicate that the request has been successfully accepted, while status codes in the 5xx range indicate internal server errors.

You can get videos url in videos of response.

Request:

curl --location --request GET 'https://api.novita.ai/v3/async/task-result?task_id=fa20dff3-18cb-4417-a7f8-269456a35154' \
--header 'Authorization: Bearer {{key}}'

Response:

{
    "task": {
        "task_id": "fa20dff3-18cb-4417-a7f8-269456a35154",
        "task_type": "TXT_TO_VIDEO",
        "status": "TASK_STATUS_SUCCEED",
        "reason": "",
        "eta": 0,
        "progress_percent": 100
    },
    "images": [],
    "videos": [
        {
            "video_url": "https://faas-output-video.s3.ap-southeast-1.amazonaws.com/test/61bc0452-03a5-4e5b-ba78-2dbd3db6cc7d/99a87dec55c6431189aff4bad39fb4a0.mp4?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIASVPYCN6LRCW3SOUV%2F20231219%2Fap-southeast-1%2Fs3%2Faws4_request&X-Amz-Date=20231219T143829Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host&x-id=GetObject&X-Amz-Signature=3b828cd8e72d9e83eb625e5e175defbfbcfc97acf4a605dc83588ae949b698b4",
            "video_url_ttl": "3600",
            "video_type": "mp4"
        }
    ]
}

Video files:

txt2video