이틀 전 Ascent GX10을 수령하였고, 지난 며칠 동안 eugr/spark-vllm-docker 이미지를 패치하여 단일 Spark에서 REAP-pruned NVFP4 DeepSeek-V4-Flash 설정을 구동하는 데 시간을 보냈습니다.
공로를 돌리자면, REAP 작업은 0xSero가 수행했습니다. 저는 단지 그것을 연결하고, 검증하고, 기계를 통해 밀어붙인 사람일 뿐입니다.
제가 확인하고 싶었던 주요 사항은 긴 컨텍스트 (long-context) 일관성이었으며, 흥미로운 부분은 컨텍스트가 확장됨에 따라 처리량 (throughput)이 얼마나 안정적으로 유지되는가 하는 점입니다.
또한, 가공되지 않은 로그에 매몰되지 않고 Spark(VLLM을 통해 262k 컨텍스트로 서비스됨)를 모니터링할 수 있도록 Hermes에서 Grafana 대시보드를 vibecoding 했습니다.
수치는 다음과 같습니다:
model test t/s (total) t/s (req) peak t/s peak t/s (req) ttfr (ms) est_ppt (ms) e2e_ttft (ms)
deepseek-v4-flash pp4092 (c1) 835.41 ± 0.00 835.41 ± 0.00 4902.67 ± 0.00 4898.18 ± 0.00 4902.67 ± 0.00
deepseek-v4-flash tg128 (c1) 23.38 ± 0.00 23.38 ± 0.00 27.00 ± 0.00 27.00 ± 0.00
deepseek-v4-flash pp4092 (c2) 544.31 ± 0.00 556.92 ± 284.68 9950.97 ± 5084.31 9946.48 ± 5084.31 9950.97 ± 5084.31
deepseek-v4-flash tg128 (c2) 16.76 ± 0.00 24.85 ± 0.63 29.00 ± 0.00 29.00 ± 0.00
deepseek-v4-flash pp4092 (c4) 458.66 ± 0.00 215.93 ± 54.18 20228.56 ± 5074.88 20224.07 ± 5074.88 20228.56 ± 5074.88
deepseek-v4-flash tg128 (c4) 14.17 ± 0.00 23.87 ± 0.75 31.00 ± 0.00 28.75 ± 1.79
deepseek-v4-flash pp4092 (c1) 827.54 ± 0.00 827.54 ± 0.00 4949.25 ± 0.00 4944.77 ± 0.00 4949.25 ± 0.00
deepseek-v4-flash tg512 (c1) 22.15 ± 0.00 22.15 ± 0.00 29.00 ± 0.00 29.00 ± 0.00
deepseek-v4-flash pp4092 (c2) 259.55 ± 0.00 483.59 ± 353.80 18211.16 ± 13320.06 18206.67 ± 13320.06 18211.16 ± 13320.06
deepseek-v4-flash tg512 (c2) 20.64 ± 0.00 22.90 ± 0.56 30.00 ± 0.00 30.00 ± 0.00
deepseek-v4-flash pp4092 (c4) 193.07 ± 0.00 105.06 ± 34.55 43677.81 ± 14362.48 43673.32 ± 14362.48 43677.81 ± 14362.48
deepseek-v4-flash tg512 (c4) 20.12 ± 0.00 23.66 ± 1.74 31.00 ± 0.00 29.50 ± 1.12
deepseek-v4-flash pp16384 (c1) 768.42 ± 0.00 768.42 ± 0.00 21326.14 ± 0.00 21321.66 ± 0.00 21328.51 ± 0.00
deepseek-v4-flash tg128 (c1) 22.14 ± 0.00 22.14 ± 0.00 27.00 ± 0.00 27.00 ± 0.00
deepseek-v4-flash pp16384 (c2) 668.24 ± 0.00 533.52 ± 199.36 35697.41 ± 13337.33 35692.92 ± 13337.33 35698.70 ± 13337.36
deepseek-v4-flash tg128 (c2) 7.83 ± 0.00 22.87 ± 0.86 28.00 ± 0.00 28.00 ± 0.00
deepseek-v4-flash pp16384 (c4) 636.72 ± 0.00 273.62 ± 59.03 62805.30 ± 13548.80 62800.81 ± 13548.80 62806.27 ± 13547.83
deepseek-v4-flash tg128 (c4) 5.81 ± 0.00 22.51 ± 1.40 28.00 ± 0.00 27.25 ± 0.83
deepseek-v4-flash pp16384 (c1) 769.23 ± 0.00 769.23 ± 0.00 21303.79 ± 0.00 21299.30 ± 0.00 21303.79 ± 0.00
deepseek-v4-flash tg512 (c1) 22.23 ± 0.00 22.23 ± 0.00 30.00 ± 0.00 30.00 ± 0.00
deepseek-v4-flash pp16384 (c2) 499.36 ± 0.00 503.44 ± 253.74 43631.21 ± 21988.43 43626.72 ± 21988.43 43631.21 ± 21988.43
deepseek-v4-flash tg512 (c2) 15.40 ± 0.00 22.65 ± 0.16 28.00 ± 0.00 28.00 ± 0.00
deepseek-v4-flash pp16384 (c4) 425.47 ± 0.00 197.99 ± 48.93 88138.11 ± 21781.16 88133.62 ± 21781.16 88138.11 ± 21781.16
deepseek-v4-flash tg512 (c4) 13.09 ± 0.00 22.30 ± 0.63 30.00 ± 0.00 29.50 ± 0.50
deepseek-v4-flash pp65536 (c1) 655.34 ± 0.00 655.34 ± 0.00 100007.10 ± 0.00 100002.61 ± 0.00 100014.84 ± 0.00
deepseek-v4-flash tg128 (c1) 18.01 ± 0.00 18.01 ± 0.00 23.00 ± 0.00 23.00 ± 0.00
deepseek-v4-flash pp65536 (c2) 622.19 ± 0.00 468.70 ± 157.58 157651.57 ± 53003.64 157647.08 ± 53003.64 157657.64 ± 53004.05
deepseek-v4-flash tg128 (c2) 2.27 ± 0.00 21.03 ± 0.62 26.00 ± 0.00 25.50 ± 0.50
deepseek-v4-flash pp65536 (c4) 613.00 ± 0.00 256.18 ± 52.33 266959.62 ± 54527.17 266955.14 ± 54527.17 266963.48 ± 54526.99
deepseek-v4-flash tg128 (c4) 1.54 ± 0.00 20.92 ± 1.06 28.00 ± 0.00 26.50 ± 0.87
deepseek-v4-flash pp65536 (c1) 656.34 ± 0.00 656.34 ± 0.00 99855.20 ± 0.00 99850.71 ± 0.00 99861.54 ± 0.00
deepseek-v4-flash tg512 (c1) 21.32 ± 0.00 21.32 ± 0.00 27.00 ± 0.00 27.00 ± 0.00
deepseek-v4-flash pp65536 (c2) 579.74 ± 0.00 462.74 ± 172.85 164598.02 ± 61483.52 164593.53 ± 61483.52 164604.29 ± 61483.75
deepseek-v4-flash tg512 (c2) 6.88 ± 0.00 20.94 ± 0.91 28.00 ± 0.00 27.50 ± 0.50
deepseek-v4-flash pp65536 (c4) 545.41 ± 0.00 234.86 ± 51.30 293034.26 ± 64009.23 293029.77 ± 64009.23 293037.88 ± 64009.22
deepseek-v4-flash tg512 (c4) 5.09 ± 0.00 21.33 ± 0.70 28.00 ± 0.00 27.50 ± 0.87
deepseek-v4-flash pp131072 (c1) 558.69 ± 0.00 558.69 ± 0.00 234608.36 ± 0.00 234603.87 ± 0.00 234621.63 ± 0.00
deepseek-v4-flash tg128 (c1) 19.10 ± 0.00 19.10 ± 0.00 23.00 ± 0.00 23.00 ± 0.00
deepseek-v4-flash pp131072 (c2) 548.87 ± 0.00 406.83 ± 132.39 360340.23 ± 117258.53 360335.75 ± 117258.53 360347.52 ± 117259.06
deepseek-v4-flash tg128 (c2) 1.05 ± 0.00 19.13 ± 0.22 25.00 ± 0.00 24.00 ± 1.00
deepseek-v4-flash pp131072 (c4) 546.73 ± 0.00 196.89 ± 56.72 602040.49 ± 121723.14 602036.01 ± 121723.14 602053.75 ± 121723.14
deepseek-v4-flash tg128 (c4) 0.70 ± 0.00 20.11 ± 1.47 25.00 ± 0.00 24.00 ± 1.22
deepseek-v4-flash pp131072 (c1) 573.71 ± 0.00 573.71 ± 0.00 228466.93 ± 0.00 228462.44 ± 0.00 228473.65 ± 0.00
deepseek-v4-flash tg512 (c1) 18.50 ± 0.00 18.50 ± 0.00 24.00 ± 0.00 24.00 ± 0.00
deepseek-v4-flash pp131072 (c2) 531.49 ± 0.00 409.53 ± 143.78 365049.44 ± 128158.79 365044.96 ± 128158.79 365059.40 ± 128161.25
deepseek-v4-flash tg512 (c2) 3.62 ± 0.00 18.88 ± 0.88 26.00 ± 0.00 25.00 ± 1.00
deepseek-v4-flash pp131072 (c4) 526.27 ± 0.00 188.42 ± 54.45 631612.72 ± 130990.99 631608.23 ± 130990.99 631626.03 ± 130991.41
deepseek-v4-flash tg512 (c4) 2.09 ± 0.00 19.28 ± 0.45 26.00 ± 0.00 25.00 ± 1.22
deepseek-v4-flash pp162816 (c1) 534.93 ± 0.00 534.93 ± 0.00 304375.99 ± 0.00 304371.51 ± 0.00 304384.97 ± 0.00
deepseek-v4-flash tg128 (c1) 20.62 ± 0.00 20.62 ± 0.00 24.00 ± 0.00 24.00 ± 0.00
deepseek-v4-flash pp162816 (c2) 521.46 ± 0.00 387.00 ± 126.26 470838.82 ± 153616.52 470834.33 ± 153616.52 470847.89 ± 153616.37
deepseek-v4-flash tg128 (c2) 0.81 ± 0.00 19.09 ± 0.42 24.00 ± 0.00 24.00 ± 0.00
deepseek-v4-flash pp162816 (c4) 519.15 ± 0.00 186.62 ± 53.53 789169.74 ± 158960.31 789165.25 ± 158960.31 789174.99 ± 158955.06
deepseek-v4-flash tg128 (c4) 0.54 ± 0.00 19.86 ± 0.79 25.00 ± 0.00 24.00 ± 1.22
deepseek-v4-flash pp162816 (c1) 542.47 ± 0.00 542.47 ± 0.00 300144.05 ± 0.00 300139.56 ± 0.00 300160.34 ± 0.00
deepseek-v4-flash tg512 (c1) 18.50 ± 0.00 18.50 ± 0.00 24.00 ± 0.00 24.00 ± 0.00
deepseek-v4-flash pp162816 (c2) 508.47 ± 0.00 388.37 ± 134.13 476007.57 ± 164392.18 476003.08 ± 164392.18 476017.56 ± 164391.67
deepseek-v4-flash tg512 (c2) 2.87 ± 0.00 17.99 ± 0.36 24.00 ± 0.00 23.00 ± 1.00
deepseek-v4-flash pp162816 (c4) 495.46 ± 0.00 207.66 ± 42.84 818907.10 ± 168931.83 818902.61 ± 168931.83 818912.38 ± 168926.54
deepseek-v4-flash tg512 (c4) 1.98 ± 0.00 18.75 ± 0.49 28.00 ± 0.00 25.25 ± 1.64
제가 주목한 점은 이 모델이 단일 Spark 환경에서 긴 컨텍스트에서도 놀라울 정도로 일관성을 유지한다는 것입니다. Prefill과 tg 수치가 4K부터 162K까지 늘어남에 따라 예상되는 것처럼 급격히 떨어지지 않았으며, 이것이 테스트의 핵심 목적이었습니다.
다음으로는 180B REAP 벤치마크도 게시할 예정이며, 하드웨어가 허락한다면 최대 500K까지 더 긴 컨텍스트를 시도해 보고 싶습니다.
제출자: /u/Dry-Tough-8068
[링크] [댓글]