안녕하세요, 저도 토큰 사양(token specs)을 게시하는 유행에 동참하고 싶습니다. CPU: 2x Intel Xeon CPU E5-2650 v4 @ 2.20GHz RAM: 2x 4 Channel 2400MHz DDR4 GPU: 1x AMD Radeon 7900 XTX 24GB 3x AMD Instinct MI60 32GB 이상한 GPU 조합이죠, 맞나요? 제가 가진 AMD Instinct MI60 32GB 중 하나가 고장 났는데, 여분이 없고 다른 선택지도 없었습니다. 프롬프트 처리(Prompt processing)는 140t/s 이상의 높은 속도를 보입니다 (60k 컨텍스트에서는 80t/s 중반까지 떨어졌습니다). 추론(Inference)은 약 11t/s입니다. llama.cpp 명령어가 최적화되지 않았습니다. llama.cpp 로그: 38.32.848.664 I slot print_timing: id 0 | task 0 | prompt processing, n_tokens = 2048, progress = 0.03, t = 14.48 s / 141.44 tokens per second 38.47.371.961 I slot print_timing: id 0 | task 0 | prompt processing, n_tokens = 4096, progress = 0.06, t = 29.00 s / 141.23 tokens per second 39.05.717.960 I slot print_timing: id 0 | task 0 | prompt processing, n_tokens = 6144, progress = 0.09, t = 47.35 s / 129.76 tokens per second .. 51.08.617.986 I slot print_timing: id 0 | task 0 | prompt processing, n_tokens = 65536, progress = 0.98, t = 770.25 s / 85.08 tokens per second 51.27.241.174 I slot print_timing: id 0 | task 0 | prompt processing, n_tokens = 66682, progress = 0.99, t = 788.87 s / 84.53 tokens per second 51.34.905.170 I slot print_timing: id 0 | task 0 | prompt processing, n_tokens = 67194, progress = 1.00, t = 796.54 s / 84.36 tokens per second 51.43.631.476 I slot print_timing: id 0 | task 0 | n_decoded = 100, tg = 11.96 t/s, tg_3s = 11.96 t/s 51.46.703.156 I slot print_timing: id 0 | task 0 | n_decoded = 135, tg = 11.81 t/s, tg_3s = 11.39 t/s 51.49.759.994 I slot print_timing: id 0 | task 0 | n_decoded = 171, tg = 11.80 t/s, tg_3s = 11.78 t/s ..

52.11.051.549 I slot print_timing: id 0 | task 0 | n_decoded = 427, tg = 11.93 t/s, tg_3s = 11.84 t/s 52.14.103.819 I slot print_timing: id 0 | task 0 | n_decoded = 464, tg = 11.95 t/s, tg_3s = 12.12 t/s 52.17.143.301 I slot print_timing: id 0 | task 0 | n_decoded = 501, tg = 11.97 t/s, tg_3s = 12.17 t/s 52.19.252.023 I slot print_timing: id 0 | task 0 | prompt eval time = 796902.50 ms / 67198 tokens ( 11.86 ms per token, 84.32 tokens per second) 52.19.252.029 I slot print_timing: id 0 | task 0 | eval time = 43980.11 ms / 526 tokens ( 83.61 ms per token, 11.96 tokens per second) llama.cpp version: 10223 (11924d4c1) llama.cpp backend: ROCm 7.2.4 llama.cpp command line: GGML_CUDA_P2P=1 llama-server -m DeepSeek-V4-Flash-0731-UD-IQ3_XXS-00001-of-00004.gguf --temp 1.0--top-p 0.95--min-p 0.00 -fa 1 -c 1048576 -np 1 --chat-template-kwargs {"reasoning_effort":"max"} -lm none -mg 0 submitted by /u/Hyungsun [link] [comments]