Image-to-Video-Motion

Base URLs:

• https://api.novita.ai

POST Image-to-Video-Motion

POST /v3/async/img2video-motion

This API generates new videos using motion videos and images.

Request header parameters

• Content-TypestringRequired

  Only application/json supported

• AuthorizationstringRequired

  In Bearer {{Key}} format.

Request Body parameters

• extraobject

  Show properties

• image_assets_idstringRequired

  image_assets_id can be get with guidance https://novita.ai/reference/video_generator/img2video-motion.html#_1-get-image-assets-id.

• motion_video_assets_idstringRequired

  motion_video_assets_id can be get with guidance https://novita.ai/reference/video_generator/img2video-motion.html#_2-get-video-assets-id. You can also use tools here: https://huggingface.co/spaces/fffiloni/video2densepose to generate your own motion videos with origin videos.

• seedintegerRequired

  Seed for video generation, Minimum: -1

Responses

• task_idstring

Example request

curl --location 'https://api.novita.ai/v3/async/img2video-motion' \ --header 'Content-Type: application/json' \ --header 'Authorization: Bearer {{key}}' \ --data '{ 'extra': { 'response_video_type': 'mp4' }, 'image_assets_id': 'cjIvbm92aXRhLWFpLWFzc2V0L2ltYWdlL1BmTlM0NFRBbmN4NlcyNXNFTXB5S3hIc3RaUVF3aGR5', 'motion_video_assets_id': 'cjIvbm92aXRhLWFpLWFzc2V0L3ZpZGVvL0dIeFhXTVF4OEpSQWpmV0hRMzU3WTNpQ2JwMjZtQWp0', 'seed': 123 }'

Response

200200

{ 'task_id': '0618e8fb-15ec-4e33-8aa0-253ddb4f07aa' }

Example

This API generates new videos using motion videos and images. The returned video can be access by API /v3/async/task-result with task_id.

Please set the Content-Type header to application/json in your HTTP request to indicate that you are sending JSON data. Currently, only JSON format is supported.

Try it in playground.

1. Get image assets id

Request:

curl -X PUT -T "{{image file path}}" 'https://assets.novitai.com/image'

Response:

{
    "assets_id":"cjIvbm92aXRhLWFpLWFzc2V0L2ltYWdlL214TVlKamtTemFCTmo3bkIyelNoWU5tSjdITVNlMjdk"
}

Image files:

2. Get video assets id

Request:

curl -X PUT -T "{{video file path}}" 'https://assets.novitai.com/video'

Response:

{
    "assets_id":"cjIvbm92aXRhLWFpLWFzc2V0L3ZpZGVvL0NLd0N3aHJwS0ZyYVduNWVoejVFV0tleGlzN0toNmRq"
}

Video files:

Motion

Below are some examples of videos and it's asset id that can be used as motion videos. You can also use tools Here to generate your own motion videos with origin videos.

cjIvbm92aXRhLWFpLWFzc2V0L3ZpZGVvL3BYTjRDTXpwNzNqa2U2Wm1Na0FiRmVpcGtXRHMyajM4

https://faas-static-assets.s3.ap-southeast-1.amazonaws.com/demo1.mp4

cjIvbm92aXRhLWFpLWFzc2V0L3ZpZGVvL1BRNlpiRW5jWW1LR1pINk1LWDViMzRCRmo1Yk1jRDY4

https://faas-static-assets.s3.ap-southeast-1.amazonaws.com/demo2.mp4

cjIvbm92aXRhLWFpLWFzc2V0L3ZpZGVvL2JrdE01UEpIY0tIU0ZmUzdINEQ0aTdTZEpTWjhBRWFC

https://faas-static-assets.s3.ap-southeast-1.amazonaws.com/demo3.mp4

cjIvbm92aXRhLWFpLWFzc2V0L3ZpZGVvL0NLd0N3aHJwS0ZyYVduNWVoejVFV0tleGlzN0toNmRq

https://faas-static-assets.s3.ap-southeast-1.amazonaws.com/demo4.mp4

cjIvbm92aXRhLWFpLWFzc2V0L3ZpZGVvL1FFcmphdGJCOFI3ODdhR0gzajVUSkJZbTNkaHIzNzRu

https://faas-static-assets.s3.ap-southeast-1.amazonaws.com/demo5.mp4

3. Request image to video with motion API

Use image and video assets id generated from step 2.1 and 2.2 to request image to video with motion API.

Request:

curl --location 'https://api.novita.ai/v3/async/img2video-motion' \
--header 'Accept: application/json' \
--header 'Authorization: Bearer {{key}}' \
--data '{
    "image_assets_id": "cjIvbm92aXRhLWFpLWFzc2V0L2ltYWdlL2J0aThqMm5EczI1bjVSYWZDNVhacGtSQ1Jud3pzUGty",
    "motion_video_assets_id": "cjIvbm92aXRhLWFpLWFzc2V0L3ZpZGVvL2hFa1B6YmhzN204UHBjU2ZBUHA4Q1hiM1h6WkJTRThC",
    "seed": 20231127
}'

Response:

{
    "task_id": "6469ca5b-5929-4200-b263-241020763eb8"
}

Use task_id to get images

HTTP status codes in the 2xx range indicate that the request has been successfully accepted, while status codes in the 5xx range indicate internal server errors.

You can get videos url in videos of response.

Request:

curl --location --request GET 'https://api.novita.ai/v3/async/task-result?task_id=6469ca5b-5929-4200-b263-241020763eb8' \
--header 'Authorization: Bearer {{key}}'

Response:

{
    "task": {
        "task_id": "6469ca5b-5929-4200-b263-241020763eb8",
        "task_type": "IMG_TO_VIDEO_MOTION",
        "status": "TASK_STATUS_SUCCEED",
        "reason": "",
        "eta": 0,
        "progress_percent": 100
    },
    "images": [],
    "videos": [
        {
            "video_url": "https://faas-output-video.s3.ap-southeast-1.amazonaws.com/test/5e39e5f0-8416-4d17-8f5f-7cbe88a5168f/033f5fc7f5734b35bedcd7cac82b59d0.mp4?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIASVPYCN6LRCW3SOUV%2F20231215%2Fap-southeast-1%2Fs3%2Faws4_request&X-Amz-Date=20231215T032655Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host&x-id=GetObject&X-Amz-Signature=53dafec708be2bbb3f814652b9a523372bc59b6334c0bf0e548109d088810701",
            "video_url_ttl": "3600",
            "video_type": "mp4"
        }
    ]
}

Motion Video files:

Motion