🧠

Model

ConsisID

Name: ConsisID
Author: Pku Yuangroup

by Pku Yuangroup gh-model--pku-yuangroup--consisid

Nexus Index

47.0 Top 100%

S: Semantic 50

A: Authority 0

P: Popularity 65

R: Recency 97

Q: Quality 50

Tech Context

Vital Performance

0 DL / 30D

0.0%

Source →

Audited 47 FNI Score

Tiny - Params

- Context

0 Downloads

Commercial APACHE License

Model Information Summary
Entity Passport
Registry ID	gh-model--pku-yuangroup--consisid
License	Apache-2.0
Provider	github

📜

Cite this model

Academic & Research Attribution

BibTeX

@misc{gh_model__pku_yuangroup__consisid,
  author = {Pku Yuangroup},
  title = {ConsisID Model},
  year = {2026},
  howpublished = {\url{https://github.com/pku-yuangroup/consisid}},
  note = {Accessed via Free2AITools Knowledge Fortress}
}

APA Style

Pku Yuangroup. (2026). ConsisID [Model]. Free2AITools. https://github.com/pku-yuangroup/consisid

🔬Technical Deep Dive

Full Specifications [+]

Quick Commands

🐙 Git Clone

git clone https://github.com/pku-yuangroup/consisid

⚖️ Nexus Index V2.0

Methodology Index Protocol

47.0

TOP 100% SYSTEM IMPACT

Semantic (S) 50

Authority (A) 0

Popularity (P) 65

Recency (R) 97

Quality (Q) 50

💬 Index Insight

FNI V2.0 for ConsisID: Semantic (S:50), Authority (A:0), Popularity (P:65), Recency (R:97), Quality (Q:50).

Free2AITools Nexus Index

Verification Authority

HuggingFace API GitHub Metadata Arxiv Citation DB System Audit

Unbiased Data Node Refresh: VFS Live

---

🚀 What's Next?

📊

Find Training Datasets

Discover datasets compatible with this model

📈

Compare Benchmarks

See how this model ranks on standard tests

⚡

Technical Deep Dive

[CVPR 2025 Highlight] Identity-Preserving Text-to-Video Generation by Frequency Decomposition

If you like our project, please give us a star ⭐ on GitHub for the latest update.

This repository is the official implementation of ConsisID, a tuning-free DiT-based controllable IPT2V model to keep human-identity consistent in the generated video. The approach draws inspiration from previous studies on frequency analysis of vision/diffusion transformers.

💡 We also have other video generation projects that may interest you ✨.

Open-Sora Plan: Open-Source Large Video Generation Model
Bin Lin, Yunyang Ge and Xinhua Cheng etc.

Helios: Real Real-Time Long Video Generation Model
Shenghai Yuan, Jinfa Huang and Xianyi He etc.

OpenS2V-Nexus: A Detailed Benchmark and Million-Scale Dataset for Subject-to-Video Generation
Shenghai Yuan, Xianyi He and Yufan Deng etc.

MagicTime: Time-lapse Video Generation Models as Metamorphic Simulators
Shenghai Yuan, Jinfa Huang and Yujun Shi etc.

ChronoMagic-Bench: A Benchmark for Metamorphic Evaluation of Text-to-Time-lapse Video Generation
Shenghai Yuan, Jinfa Huang and Yongqi Xu etc.

📣 News

⏳⏳⏳ Release the full code & datasets & weights.
[2026.03.08] 👋 We introduce Helios, a breakthrough video generation model that achieves minute-scale, high-quality video synthesis at 19.5 FPS on a single H100 GPU — without relying on conventional long video anti-drifting strategies or standard video acceleration techniques. Welcome to check Technical Report!
[2025.08.30] 🚀 Thanks for the excellent work DanceGRPO on transferring ConsisID data for I2V RL training, please refer to here for more details. Similarly, you can also try using OpenS2V-5M for RL training.
[2025.05.27] 🔥 Introducing OpenS2V-Nexus, which consists of: (i) OpenS2V-Eval, a fine-grained benchmark, and (ii) OpenS2V-5M, a million-scale dataset. Welcome to try it!
[2025.04.04] 🔥 Breaking news! ConsisID has been recommended as CVPR Highlight.
[2025.03.27] 🔥 We have updated our technical report. Please click here to view it.
[2025.02.27] 🔥 ConsisID has been accepted by CVPR 2025, and we will update arXiv with more details soon. Stay tuned!
[2025.02.16] 🔥 We have adapted the code for CogVideoX1.5, and you can use our code not only for training ConsisID but also for the CogVideoX-series.
[2025.01.19] 🤗 Thanks @arrow, @yiyixuxu, @hlky and @stevhliu, ConsisID will be merged into diffusers in 0.33.0. So for now, please use pip install git+https://github.com/huggingface/diffusers.git to install diffusers dev version. And we have reorganized the code and weight configs, so it's better to update your local files if you have cloned them previously.
[2024.12.26] 🚀 We release the cache inference code for ConsisID powered by TeaCache. Thanks @LiewFeng for his help.
[2024.12.24] 🚀 We release the parallel inference code for ConsisID powered by xDiT. Thanks @feifeibear for his help.
[2024.12.09] 🔥We release the test set and metric calculation code used in the paper, now your can measure the metrics on your own machine. Please refer to this guide for more details.
[2024.12.08] 🔥The code for data preprocessing is out, which is used to obtain the training data required by ConsisID, supporting multi-id annotation. Please refer to this guide for more details.
[2024.12.04] Thanks @shizi for providing 🤗Windows-ConsisID and 🟣Windows-ConsisID, which make it easy to run ConsisID on Windows.
[2024.12.01] 🔥 We provide full text prompts corresponding to all the videos on project page. Click here to get and try the demo.
[2024.11.30] 🤗 We have fixed the huggingface demo, welcome to try it.
[2024.11.29] 🏃‍♂️ The current code and weights are our early versions, and the differences with the latest version in arxiv can be viewed here. And we will release the full code in the next few days.
[2024.11.28] Thanks @camenduru for providing Jupyter Notebook and @Kijai for providing ComfyUI Extension ComfyUI-ConsisIDWrapper. If you find related work, please let us know.
[2024.11.27] 🏃‍♂️ Due to policy restrictions, we only open-source part of the dataset. You can download it by clicking here. And we will release the data processing code in the next few days.
[2024.11.26] 🔥 We release the arXiv paper for ConsisID, and you can click here to see more details.
[2024.11.22] 🔥 All code & datasets are coming soon! Stay tuned 👀!

😍 Gallery

Identity-Preserving Text-to-Video Generation. (Some best prompts here)

or you can click here to watch the video.

🤗 Demo

Diffusers API

bash

!pip install diffusers==0.33.1
import torch
from diffusers import ConsisIDPipeline
from diffusers.pipelines.consisid.consisid_utils import prepare_face_models, process_face_embeddings_infer
from diffusers.utils import export_to_video
from huggingface_hub import snapshot_download

snapshot_download(repo_id="BestWishYsh/ConsisID-preview", local_dir="BestWishYsh/ConsisID-preview")
face_helper_1, face_helper_2, face_clip_model, face_main_model, eva_transform_mean, eva_transform_std = (
    prepare_face_models("BestWishYsh/ConsisID-preview", device="cuda", dtype=torch.bfloat16)
)
pipe = ConsisIDPipeline.from_pretrained("BestWishYsh/ConsisID-preview", torch_dtype=torch.bfloat16)
pipe.to("cuda")

# ConsisID works well with long and well-described prompts. Make sure the face in the image is clearly visible (e.g., preferably half-body or full-body).
prompt = "The video captures a boy walking along a city street, filmed in black and white on a classic 35mm camera. His expression is thoughtful, his brow slightly furrowed as if he's lost in contemplation. The film grain adds a textured, timeless quality to the image, evoking a sense of nostalgia. Around him, the cityscape is filled with vintage buildings, cobblestone sidewalks, and softly blurred figures passing by, their outlines faint and indistinct. Streetlights cast a gentle glow, while shadows play across the boy's path, adding depth to the scene. The lighting highlights the boy's subtle smile, hinting at a fleeting moment of curiosity. The overall cinematic atmosphere, complete with classic film still aesthetics and dramatic contrasts, gives the scene an evocative and introspective feel."
image = "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/consisid/consisid_input.png?download=true"

id_cond, id_vit_hidden, image, face_kps = process_face_embeddings_infer(
    face_helper_1,
    face_clip_model,
    face_helper_2,
    eva_transform_mean,
    eva_transform_std,
    face_main_model,
    "cuda",
    torch.bfloat16,
    image,
    is_align_face=True,
)

video = pipe(
    image=image,
    prompt=prompt,
    num_inference_steps=50,
    guidance_scale=6.0,
    use_dynamic_cfg=False,
    id_vit_hidden=id_vit_hidden,
    id_cond=id_cond,
    kps_cond=face_kps,
    generator=torch.Generator("cuda").manual_seed(42),
)
export_to_video(video.frames[0], "output.mp4", fps=8)

Gradio Web UI

Highly recommend trying out our web demo by the following command, which incorporates all features currently supported by ConsisID. We also provide online demo in Hugging Face Spaces.

bash

python app.py

CLI Inference

bash

python infer.py --model_path BestWishYsh/ConsisID-preview

warning: It is worth noting that even if we use the same seed and prompt but we change a machine, the results will be different.

Prompt Refiner

ConsisID has high requirements for prompt quality. You can use GPT-4o to refine the input text prompt, an example is as follows (original prompt: "a man is playing guitar.")

bash

a man is playing guitar.

Change the sentence above to something like this (add some facial changes, even if they are minor. Don't make the sentence too long): 

The video features a man standing next to an airplane, engaged in a conversation on his cell phone. he is wearing sunglasses and a black top, and he appears to be talking seriously. The airplane has a green stripe running along its side, and there is a large engine visible behind his. The man seems to be standing near the entrance of the airplane, possibly preparing to board or just having disembarked. The setting suggests that he might be at an airport or a private airfield. The overall atmosphere of the video is professional and focused, with the man's attire and the presence of the airplane indicating a business or travel context.

Some sample prompts are available here.

GPU Memory Optimization

ConsisID requires about 44 GB of GPU memory to decode 49 frames (6 seconds of video at 8 FPS) with output resolution 720x480 (W x H), which makes it not possible to run on consumer GPUs or free-tier T4 Colab. The following memory optimizations could be used to reduce the memory footprint. For replication, you can refer to this script.

Feature (overlay the previous)	Max Memory Allocated	Max Memory Reserved
-	37 GB	44 GB
enable_model_cpu_offload	22 GB	25 GB
enable_sequential_cpu_offload	16 GB	22 GB
vae.enable_slicing	16 GB	22 GB
vae.enable_tiling	5 GB	7 GB

bash

# turn on if you don't have multiple GPUs or enough GPU memory(such as H100)
pipe.enable_model_cpu_offload()
pipe.enable_sequential_cpu_offload()
pipe.vae.enable_slicing()
pipe.vae.enable_tiling()

warning: it will cost more time in inference and may also reduce the quality.

🚀 Parallel Inference on Multiple GPUs by xDiT

xDiT is a Scalable Inference Engine for Diffusion Transformers (DiTs) on multi-GPU Clusters. It has successfully provided low-latency parallel inference solutions for a variety of DiTs models. For example, to generate a video with 6 GPUs, you can use the following command:

text

cd tools/parallel_inference
bash run.sh
# run_usp.sh

🚀 Cache Inference by TeaCache

TeaCache is a training-free caching approach that estimates and leverages the fluctuating differences among model outputs across timesteps, thereby accelerating the inference. For example, you can use the following command:

text

cd tools/cache_inference
bash run.sh

⚙️ Requirements and Installation

We recommend the requirements as follows.

Environment

bash

# 0. Clone the repo
git clone --depth=1 https://github.com/PKU-YuanGroup/ConsisID.git
cd ConsisID

# 1. Create conda environment
conda create -n consisid python=3.11.0
conda activate consisid

# 3. Install PyTorch and other dependencies using conda
# CUDA 11.8
conda install pytorch==2.5.1 torchvision==0.20.1 torchaudio==2.5.1 pytorch-cuda=11.8 -c pytorch -c nvidia
# CUDA 12.1
conda install pytorch==2.5.1 torchvision==0.20.1 torchaudio==2.5.1 pytorch-cuda=12.1 -c pytorch -c nvidia

# 4. Install pip dependencies
pip install -r requirements.txt

Download ConsisID

The weights are available at 🤗HuggingFace, 🤖ModelScope and 🟣WiseModel, and will be automatically downloaded when runing app.py and infer.py, or you can download it with the following commands.

bash

# way 1
# if you are in china mainland, run this first: export HF_ENDPOINT=https://hf-mirror.com
cd util
python download_weights.py

# way 2
# if you are in china mainland, run this first: export HF_ENDPOINT=https://hf-mirror.com
huggingface-cli download --repo-type model \
BestWishYsh/ConsisID-preview \
--local-dir ckpts

# way 3
modelscope download --model \
BestWishYSH/ConsisID-preview \
--local-dir ckpts

# way 4
git lfs install
git clone https://www.wisemodel.cn/SHYuanBest/ConsisID-Preview.git

Once ready, the weights will be organized in this format:

text

📦 ckpts/
├── 📂 data_process/
├── 📂 face_encoder/
├── 📂 scheduler/
├── 📂 text_encoder/
├── 📂 tokenizer/
├── 📂 transformer/
├── 📂 vae/
├── 📄 configuration.json
├── 📄 model_index.json

🗝️ Training

Data preprocessing

Please refer to this guide for how to obtain the training data required by ConsisID. If you want to train your own identity-preserving text-to-video generation model, you need to arrange all the dataset in this format:

text

📦 datasets/
├── 📂 captions/
│   ├── 📄 dataname_1.json
│   ├── 📄 dataname_2.json
├── 📂 dataname_1/
│   ├── 📂 refine_bbox_jsons/
│   ├── 📂 track_masks_data/
│   ├── 📂 videos/
├── 📂 dataname_2/
│   ├── 📂 refine_bbox_jsons/
│   ├── 📂 track_masks_data/
│   ├── 📂 videos/
├── ...
├── 📄 total_train_data.txt

Video DiT training

First, setting hyperparameters:

environment (e.g., cuda): deepspeed_configs
training arguments (e.g., batchsize): train_single_rank.sh or train_multi_rank.sh

Then, we run the following bash to start training:

bash

# For single rank
bash train_single_rank.sh
# For multi rank
bash train_multi_rank.sh

🙌 Friendly Links

We found some plugins created by community developers. Thanks for their efforts:

ComfyUI Extension. ComfyUI-ConsisIDWrapper (by @Kijai).
Jupyter Notebook. Jupyter-ConsisID (by @camenduru).
Windows Docker. 🤗Windows-ConsisID and 🟣Windows-ConsisID (by @shizi).
Diffusres. Diffusers-ConsisID (thanks @arrow, @yiyixuxu, @hlky and @stevhliu for their help).
xDiT. xDiT-ConsisID (thanks @feifeibear for his help).
TeaCache. TeaCache-ConsisID (thanks @LiewFeng for his help).
Ingredients: A powerful way to customize video creations by incorporating multiple specific identity (ID) photos, based on ConsisID.

If you find related work, please let us know.

🐳 Dataset

We release the subset of the data used to train ConsisID. The dataset is available at HuggingFace, or you can download it with the following command. Some samples can be found on our Project Page.

bash

huggingface-cli download --repo-type dataset \
BestWishYsh/ConsisID-preview-Data \
--local-dir BestWishYsh/ConsisID-preview-Data

🛠️ Evaluation

We release the data used for evaluation in ConsisID, which is available at HuggingFace. Please refer to this guide for how to evaluate customized model.

👍 Acknowledgement

This project wouldn't be possible without the following open-sourced repositories: Open-Sora Plan, CogVideoX, EasyAnimate, CogVideoX-Fun, IP-Adapter, PhotoMaker, UniPortrait.

🔒 License

The majority of this project is released under the Apache 2.0 license as found in the LICENSE file.
The CogVideoX-5B model (Transformers module) is released under the CogVideoX LICENSE.
The service is a research preview. Please contact us if you find any potential violations. (shyuan-cs@hotmail.com)

✏️ Citation

If you find our paper and code useful in your research, please consider giving a star :star: and citation :pencil:.

BibTeX

@inproceedings{yuan2025identity,
  title={Identity-preserving text-to-video generation by frequency decomposition},
  author={Yuan, Shenghai and Huang, Jinfa and He, Xianyi and Ge, Yunyang and Shi, Yujun and Chen, Liuhan and Luo, Jiebo and Yuan, Li},
  booktitle={Proceedings of the Computer Vision and Pattern Recognition Conference},
  pages={12978--12988},
  year={2025}
}

🤝 Contributors

🚀 Quick Start

bash

# 0. Clone the repo
git clone --depth=1 https://github.com/PKU-YuanGroup/ConsisID.git
cd ConsisID

# 1. Create conda environment
conda create -n consisid python=3.11.0
conda activate consisid

# 3. Install PyTorch and other dependencies using conda
# CUDA 11.8
conda install pytorch==2.5.1 torchvision==0.20.1 torchaudio==2.5.1 pytorch-cuda=11.8 -c pytorch -c nvidia
# CUDA 12.1
conda install pytorch==2.5.1 torchvision==0.20.1 torchaudio==2.5.1 pytorch-cuda=12.1 -c pytorch -c nvidia

# 4. Install pip dependencies
pip install -r requirements.txt

⚠️ Incomplete Data

Some information about this model is not available. Use with Caution - Verify details from the original source before relying on this data.

View Original Source →

📝 Limitations & Considerations

• Benchmark scores may vary based on evaluation methodology and hardware configuration.
• VRAM requirements are estimates; actual usage depends on quantization and batch size.
• FNI scores are relative rankings and may change as new models are added.
⚠ License Unknown: Verify licensing terms before commercial use.

Social Proof

GitHub Repository

833Stars

Repo Issues

🐙 Data Source: GitHub ↗

🔄 Daily sync (03:00 UTC)

AI Summary: Based on GitHub metadata. Not a recommendation.

📊 FNI Methodology 📚 Knowledge Baseℹ️ Verify with original source

🛡️ Model Transparency Report

Technical metadata sourced from upstream repositories.

Open Metadata

🆔 Identity & Source

id: gh-model--pku-yuangroup--consisid
slug: pku-yuangroup--consisid
source: github
author: Pku Yuangroup
license: Apache-2.0
tags: diffusion, diffusion-models, text-to-video, video-generation, identity-preserving, video-generator, videogeneration, video-generation-dataset, python

⚙️ Technical Specs

architecture: null
params billions: null
context length: null
pipeline tag: text-to-image

📊 Engagement & Metrics

downloads: 0
stars: 833
forks: 0

Data indexed from public sources. Updated daily.

Welcome to Free2AI Tools!

Smart Search

FNI Score

You're All Set!

Cite this model

🔬Technical Deep Dive

Quick Commands

⚖️ Nexus Index V2.0

💬 Index Insight

Verification Authority

🚀 What's Next?

Find Training Datasets

Compare Benchmarks

Deployment Guide

Technical Deep Dive

[CVPR 2025 Highlight] Identity-Preserving Text-to-Video Generation by Frequency Decomposition

If you like our project, please give us a star ⭐ on GitHub for the latest update.

📣 News

😍 Gallery

🤗 Demo

Diffusers API

Gradio Web UI

CLI Inference

Prompt Refiner

GPU Memory Optimization

🚀 Parallel Inference on Multiple GPUs by xDiT

🚀 Cache Inference by TeaCache

⚙️ Requirements and Installation

Environment

Download ConsisID

🗝️ Training

Data preprocessing

Video DiT training

🙌 Friendly Links

🐳 Dataset

🛠️ Evaluation

👍 Acknowledgement

🔒 License

✏️ Citation

🤝 Contributors

🚀 Quick Start

⚠️ Incomplete Data

📝 Limitations & Considerations

Social Proof

🛡️ Model Transparency Report

🆔 Identity & Source

⚙️ Technical Specs

📊 Engagement & Metrics