Model | Overall | Basic Following | Advanced Following | Designer | ||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Avg | Attribute | Relation | Reasoning | Avg | Attribute +Relation |
Attribute +Reasoning |
Relation +Reasoning |
Style | Text | Real World | ||||||||||||||
short | long | short | long | short | long | short | long | short | long | short | long | short | long | short | long | short | long | short | long | short | long | short | long | |
Diffusion based Models (GPT-4o Evaluation) | ||||||||||||||||||||||||
FLUX.1 dev | 71.17 | 71.78 | 83.23 | 78.65 | 87.17 | 83.17 | 87.39 | 80.39 | 75.14 | 72.39 | 65.79 | 68.54 | 67.07 | 73.69 | 73.84 | 73.34 | 69.09 | 71.59 | 66.67 | 66.67 | 43.83 | 52.83 | 70.72 | 71.47 |
SD XL | 54.96 | 42.13 | 65.72 | 53.28 | 59.33 | 50.83 | 77.57 | 62.57 | 60.32 | 46.57 | 49.73 | 36.22 | 47.82 | 35.57 | 56.22 | 45.34 | 52.59 | 36.09 | 73.33 | 60.00 | 16.83 | 0.83 | 50.92 | 41.59 |
SD 3 | 67.46 | 66.09 | 78.32 | 77.75 | 83.33 | 79.83 | 82.07 | 78.82 | 71.07 | 74.07 | 61.46 | 59.56 | 61.07 | 64.07 | 68.84 | 70.34 | 50.96 | 57.84 | 66.67 | 76.67 | 59.83 | 20.83 | 63.23 | 67.34 |
SD 3.5 | 71.15 | 66.96 | 78.34 | 79.56 | 79.50 | 76.50 | 80.96 | 83.21 | 72.46 | 78.71 | 67.67 | 61.18 | 66.46 | 61.89 | 73.53 | 74.15 | 60.03 | 61.53 | 73.33 | 63.33 | 70.52 | 42.52 | 64.43 | 66.39 |
SANA Sprint | 63.68 | 58.50 | 76.58 | 71.00 | 75.33 | 71.33 | 81.82 | 72.07 | 72.57 | 69.57 | 57.67 | 51.80 | 55.32 | 54.94 | 68.46 | 66.72 | 62.59 | 63.46 | 80.00 | 60.00 | 8.83 | 5.83 | 66.96 | 58.01 |
SANA 1.5 | 67.15 | 65.73 | 79.66 | 77.08 | 79.83 | 77.83 | 85.57 | 83.57 | 73.57 | 69.82 | 61.50 | 60.67 | 65.32 | 56.57 | 69.96 | 73.09 | 62.96 | 65.84 | 80.00 | 80.00 | 17.83 | 15.83 | 71.07 | 68.83 |
Playground v2 | 45.64 | 52.78 | 59.83 | 69.58 | 51.33 | 66.33 | 70.57 | 76.07 | 57.57 | 66.32 | 38.43 | 44.75 | 41.57 | 45.57 | 48.96 | 59.97 | 41.72 | 53.84 | 53.33 | 60.00 | 0.00 | 0.83 | 45.32 | 46.44 |
Playground v2.5 | 47.73 | 54.82 | 63.08 | 68.08 | 57.83 | 73.83 | 71.82 | 77.32 | 59.57 | 53.07 | 40.73 | 48.17 | 39.70 | 45.82 | 49.59 | 64.22 | 44.22 | 46.72 | 60.00 | 80.00 | 0.00 | 4.83 | 47.19 | 47.56 |
PixArt-delta | 41.01 | 48.24 | 53.83 | 59.25 | 46.33 | 52.83 | 62.07 | 71.32 | 53.07 | 53.57 | 34.60 | 42.77 | 32.44 | 37.44 | 53.59 | 56.59 | 36.96 | 49.46 | 46.67 | 73.33 | 0.00 | 0.00 | 38.23 | 40.10 |
PixArt-alpha | 44.37 | 50.50 | 55.50 | 61.00 | 52.33 | 56.33 | 63.82 | 74.07 | 50.32 | 52.57 | 38.71 | 44.90 | 37.82 | 41.32 | 58.84 | 52.46 | 40.22 | 47.09 | 50.00 | 76.67 | 0.00 | 0.83 | 45.70 | 53.16 |
PixArt-sigma | 62.00 | 58.12 | 70.66 | 75.25 | 69.33 | 78.83 | 75.07 | 77.32 | 67.57 | 69.57 | 57.65 | 49.50 | 65.20 | 56.57 | 66.96 | 61.72 | 66.59 | 54.59 | 83.33 | 70.00 | 1.83 | 1.83 | 62.11 | 52.41 |
LUMINA-Next | 50.93 | 52.46 | 64.58 | 66.08 | 56.83 | 59.33 | 67.57 | 71.82 | 69.32 | 67.07 | 44.75 | 45.63 | 51.44 | 43.20 | 51.09 | 59.72 | 44.72 | 54.46 | 70.00 | 66.67 | 0.00 | 0.83 | 47.56 | 49.05 |
Hunyuan-DiT | 51.38 | 53.28 | 69.33 | 69.00 | 65.83 | 69.83 | 78.07 | 73.82 | 64.07 | 63.32 | 42.62 | 45.45 | 50.20 | 41.57 | 59.22 | 61.84 | 47.84 | 51.09 | 56.67 | 73.33 | 0.00 | 0.83 | 40.10 | 44.20 |
AR based Models (GPT-4o Evaluation) | ||||||||||||||||||||||||
Llamagen | 41.67 | 38.22 | 53.00 | 50.00 | 48.33 | 42.33 | 59.57 | 60.32 | 51.07 | 47.32 | 35.89 | 32.61 | 38.82 | 31.57 | 40.84 | 47.22 | 49.59 | 46.22 | 46.67 | 33.33 | 0.00 | 0.00 | 39.73 | 35.62 |
LightGen | 53.22 | 43.41 | 66.58 | 47.91 | 55.83 | 47.33 | 74.82 | 45.82 | 69.07 | 50.57 | 46.74 | 41.53 | 62.44 | 40.82 | 61.71 | 50.47 | 50.34 | 45.34 | 53.33 | 53.33 | 0.00 | 6.83 | 50.92 | 50.55 |
Show-o | 59.72 | 58.86 | 73.08 | 75.83 | 74.83 | 79.83 | 78.82 | 78.32 | 65.57 | 69.32 | 53.67 | 50.38 | 60.95 | 56.82 | 68.59 | 68.96 | 66.46 | 56.22 | 63.33 | 66.67 | 3.83 | 2.83 | 55.02 | 50.92 |
Infinity | 62.07 | 62.32 | 73.08 | 75.41 | 74.33 | 76.83 | 72.82 | 77.57 | 72.07 | 71.82 | 56.64 | 54.98 | 60.44 | 55.57 | 74.22 | 64.71 | 60.22 | 59.71 | 80.00 | 73.33 | 10.83 | 23.83 | 54.28 | 56.89 |
JanusPro | 66.50 | 65.02 | 79.33 | 78.25 | 79.33 | 82.33 | 78.32 | 73.32 | 80.32 | 79.07 | 59.71 | 58.82 | 66.07 | 56.20 | 70.46 | 70.84 | 67.22 | 59.97 | 60.00 | 70.00 | 28.83 | 33.83 | 65.84 | 60.25 |
Closed-Source Models (GPT-4o Evaluation) | ||||||||||||||||||||||||
DALL-E 3 | 74.96 | 70.81 | 78.72 | 78.50 | 79.50 | 79.83 | 80.82 | 78.82 | 75.82 | 76.82 | 73.39 | 67.27 | 73.45 | 67.20 | 72.01 | 71.34 | 63.59 | 60.72 | 89.66 | 86.67 | 66.83 | 54.83 | 72.93 | 60.99 |
MidJourney v6 | 70.78 | 67.70 | 76.00 | 69.08 | 77.83 | 69.33 | 81.32 | 73.07 | 68.82 | 64.82 | 68.54 | 67.62 | 57.82 | 61.95 | 69.84 | 63.96 | 57.46 | 60.34 | 83.33 | 73.33 | 75.83 | 73.83 | 65.10 | 68.46 |
MidJourney v7 | 68.74 | 65.69 | 77.41 | 76.00 | 77.58 | 81.83 | 82.07 | 76.82 | 72.57 | 69.32 | 64.66 | 60.53 | 67.20 | 62.70 | 81.22 | 71.59 | 60.72 | 64.59 | 83.33 | 80.00 | 24.83 | 20.83 | 68.83 | 63.61 |
FLUX.1 Pro | 67.32 | 69.89 | 79.08 | 78.91 | 78.83 | 81.33 | 82.82 | 83.82 | 75.57 | 71.57 | 61.10 | 65.37 | 62.32 | 65.57 | 69.84 | 71.47 | 65.96 | 67.72 | 63.00 | 63.00 | 35.83 | 55.83 | 71.80 | 68.80 |
GPT-4o | 89.15 | 88.29 | 90.75 | 89.66 | 91.33 | 87.08 | 84.57 | 84.57 | 96.32 | 97.32 | 88.55 | 88.35 | 87.07 | 89.44 | 87.22 | 83.96 | 85.59 | 83.21 | 90.00 | 93.33 | 89.83 | 86.83 | 89.73 | 93.46 |