Model | Overall | Basic Following | Advanced Following | Designer | ||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Avg | Attribute | Relation | Reasoning | Avg | Attribute +Relation |
Attribute +Reasoning |
Relation +Reasoning |
Style | Text | Real World | ||||||||||||||
short | long | short | long | short | long | short | long | short | long | short | long | short | long | short | long | short | long | short | long | short | long | short | long | |
Diffusion based Models (GPT-4o Evaluation) | ||||||||||||||||||||||||
FLUX.1 dev | 63.47 | 67.32 | 70.03 | 71.84 | 77.75 | 78.80 | 78.02 | 78.67 | 54.31 | 58.04 | 62.69 | 65.11 | 66.95 | 69.74 | 60.99 | 60.75 | 62.47 | 64.66 | 63.33 | 72.67 | 43.25 | 58.91 | 64.14 | 63.65 |
SD 3 | 66.83 | 63.69 | 70.23 | 69.73 | 80.05 | 78.15 | 76.14 | 74.83 | 54.50 | 56.22 | 64.96 | 64.62 | 72.88 | 72.48 | 59.44 | 61.39 | 62.10 | 63.15 | 75.33 | 77.00 | 58.24 | 26.87 | 62.78 | 63.15 |
SD 3.5 | 69.59 | 64.96 | 71.74 | 70.07 | 80.25 | 78.35 | 78.61 | 75.61 | 56.38 | 56.24 | 66.90 | 62.69 | 73.41 | 68.33 | 60.73 | 57.30 | 64.51 | 62.95 | 78.33 | 71.67 | 71.79 | 49.76 | 62.28 | 64.39 |
SANA 1.5 | 65.17 | 62.17 | 70.18 | 69.03 | 76.85 | 75.25 | 78.73 | 77.60 | 54.97 | 54.23 | 65.75 | 62.45 | 72.42 | 68.66 | 63.84 | 61.76 | 63.69 | 60.66 | 93.00 | 81.00 | 16.91 | 14.13 | 66.13 | 66.25 |
Playground v2 | 46.30 | 54.63 | 53.10 | 63.31 | 55.75 | 67.00 | 63.44 | 73.87 | 40.11 | 49.06 | 44.97 | 54.82 | 50.90 | 63.34 | 39.48 | 49.99 | 46.95 | 54.37 | 69.33 | 81.67 | 1.10 | 2.16 | 49.63 | 50.25 |
Playground v2.5 | 46.34 | 54.04 | 54.33 | 63.04 | 57.75 | 69.25 | 63.32 | 74.07 | 41.90 | 45.79 | 45.41 | 54.31 | 53.14 | 63.90 | 39.69 | 49.52 | 46.40 | 52.45 | 65.33 | 80.00 | 1.48 | 5.08 | 48.01 | 46.28 |
PixArt-delta | 43.32 | 48.92 | 50.62 | 55.52 | 50.55 | 54.90 | 62.86 | 68.07 | 38.44 | 43.60 | 42.05 | 47.37 | 47.45 | 51.38 | 37.40 | 43.00 | 43.50 | 48.74 | 66.33 | 85.67 | 0.19 | 1.15 | 43.18 | 43.80 |
PixArt-alpha | 45.83 | 51.12 | 52.27 | 57.87 | 54.95 | 58.75 | 63.02 | 70.34 | 38.83 | 44.53 | 45.17 | 50.04 | 52.79 | 56.62 | 39.76 | 45.80 | 45.91 | 49.27 | 66.00 | 86.00 | 0.72 | 1.39 | 50.50 | 47.39 |
PixArt-sigma | 58.17 | 58.80 | 64.88 | 67.21 | 70.30 | 75.00 | 71.59 | 74.88 | 52.75 | 51.75 | 58.80 | 58.09 | 65.13 | 65.74 | 56.08 | 54.86 | 58.28 | 56.80 | 89.33 | 87.33 | 3.45 | 3.88 | 56.58 | 58.93 |
LUMINA-Next | 50.37 | 49.83 | 57.32 | 57.49 | 61.20 | 60.85 | 64.93 | 66.54 | 45.83 | 45.08 | 50.27 | 48.23 | 55.25 | 54.13 | 45.84 | 43.07 | 52.55 | 49.70 | 77.00 | 77.67 | 1.01 | 0.96 | 49.75 | 50.50 |
Hunyuan-DiT | 50.22 | 53.63 | 61.92 | 64.57 | 66.85 | 71.35 | 70.92 | 73.45 | 48.00 | 48.92 | 52.86 | 52.94 | 61.37 | 59.52 | 49.99 | 48.51 | 53.41 | 53.71 | 55.67 | 81.67 | 0.62 | 1.01 | 45.16 | 44.54 |
AR based Models (GPT-4o Evaluation) | ||||||||||||||||||||||||
Llamagen | 40.40 | 39.85 | 48.42 | 50.75 | 45.70 | 54.15 | 60.96 | 58.56 | 38.58 | 39.55 | 39.75 | 38.77 | 42.34 | 38.69 | 37.48 | 37.57 | 42.58 | 43.87 | 53.00 | 45.00 | 1.10 | 1.96 | 41.81 | 39.33 |
LightGen | 52.77 | 46.31 | 61.49 | 51.68 | 65.90 | 58.80 | 69.26 | 54.94 | 49.30 | 41.31 | 54.01 | 46.14 | 59.16 | 48.93 | 52.38 | 45.41 | 55.29 | 47.55 | 67.00 | 58.00 | 2.63 | 6.47 | 53.97 | 55.33 |
Show-o | 56.94 | 58.17 | 66.28 | 69.56 | 72.20 | 78.30 | 74.08 | 76.52 | 52.56 | 53.85 | 59.46 | 58.76 | 66.92 | 66.38 | 56.03 | 55.72 | 61.29 | 59.31 | 68.67 | 72.00 | 3.16 | 4.65 | 57.57 | 56.82 |
Infinity | 63.11 | 60.83 | 69.55 | 67.02 | 75.70 | 76.15 | 78.18 | 74.27 | 54.77 | 50.63 | 64.58 | 60.95 | 70.98 | 68.94 | 62.25 | 58.42 | 63.87 | 58.33 | 81.00 | 79.67 | 21.17 | 19.54 | 60.05 | 61.54 |
Janus-Pro | 64.41 | 62.19 | 70.97 | 68.15 | 77.10 | 76.55 | 76.84 | 76.45 | 58.98 | 51.45 | 64.93 | 62.87 | 71.76 | 71.59 | 61.88 | 59.57 | 64.48 | 60.46 | 71.00 | 68.33 | 32.14 | 33.19 | 65.51 | 62.16 |