Video: 엔비디아 독점 인터뷰 | 블랙웰 GPU 핵심 설계 리더가 직접 밝힌 AI 팩토리와 반도체 시장의 미래
Channel: 안될공학 - IT 테크 신기술
Duration: 19m 42s
Source: subtitle (auto, ko)
Transcript:

So fundamentally Vera is the CPU for the age ofic AI and multiple.네 네 여러분 반갑습니다. 여러입니다. 지금 제가 GTC 타이페이 와서 여기서 직접 만나뵐 수 있는 엔비디아의 핵심 인무를 만날 수 있게 되었는데요. 지금 루빈 GPU의 출시를 앞두고 있는 상황에서 현재로서 역사상 가장 강력한 AI 칩으로 평가받는 블랙엘의 출시를 직접 현장에서 진의한 임무를 만나게 되었습니다. 바로 샤르 나라심만 디렉터 이분이신데요. 엔비디아에서 AI와 데이터 센터 GPU 제품 마케팅 총괄 디렉터입니다. 이게 지금 엔비디아가 단순한 반도체이 팬리스 기업을 넘어 가지고 전 세계 AI와 관련된 모든 인프라를 바꾸고 있는 상황인데 M비디아의 진짜 전략이 무엇인지 앞으로 AI 시대의 미래를 바꿀이 인프라와 칩셋의 비밀을 지금부터 하나씩 살펴볼 건데요. 이번 GTC 타이 키노트에서도 무엇을 강조했느냐? 과거에는 단순히 얼마나 빠른 더 빠른 GPU인가를 따졌다라고 하면은 이제 시장의 질문이 달라졌죠.

AI의 대량의 데이터를 스스로 처리하고 가치를 뭔가 만들어내는 엔비디아가 얘기하는 일종의 AI 팩토리라는 시대가 도래를 하면서 이제 고객들이 토큰당 생성 비용이랑 와트당 효율성 같은 철자이 상업적 가치에 주목을 하기 시작했다라는 거죠. 여기에 대해서 직접 물어보았습니다. AI factory not as infrastructure that create values. So from a market perspective customers seem to be looking faster GPUs and asking how efficiently they can produce and monetize tokens. So from a data center GPU perspective how are customers changing the way they evaluate AI factory investments and should we now think less in terms of big performance and more in terms of cost per token, tokens per wat and GPU utilization?

How how much? So the most important metric that customers look at is how do they minimize the token cost? They want the lowest token cost possible. Now, in Jenson's keynote, he actually gave a chart where he showed you sort of the relationship between how quickly you can get your AI factory stood up, how many tokens it can actually generate, which uh directly corresponds to your revenue. Because right now, anyone who can sell a token, anyone who can create a token can automatically sell a token because there is so much demand for for AI tokens.

And you want to minimize your token cost because that maximizes your profit. And then the last thing you look at is what is the longevity of that AI factory? Right now we see even uh cloud instances of hopper GPUs in very high demand. So the things that customers most look minimize their token cost but right behind that are the considerations for how quickly can they stand up the data center or the AI factory and following our reference designs enables them to do that. How many tokens can they produce because that directly translates into their revenues.

And since we have the most performant platform, we can actually generate the most tokens and maximize their revenues and th their profits. And th and also because we constantly maintain and speed up our infrastructure with software improvements, we also allow you to extend the lifetime and lifespan of that AI factory. So on all of these fronts, Nvidia is really focus on maximizing what the customers are able to receive. 자, AI 기술이 이제 지시에 답을 하는 수준을 넘어 가지고 스스로 계획을 세우고 코드를 실행하고 뭐 결과까지 도출하고 이런 에이전틱 AI로 진화를 하고 있는 상황인데 사실이 말이 AI 연상 과정이 그냥 단순하게 한번 물어보고 답하는 이런 1회성 답변이 아니라 진짜 매우 길고 복잡한 어떤 워크플로우로 바뀐다라는 걸 얘기를 하는 거잖아.

이게 핵심이거든요. 당연히 데이터 센터 내부의 GPU, CPU 그리고 네트워크 간의 데이터 이동 방식 자체가 완전히 달라져야 한다라는게 이번 키노트에서도 확실히 강조가 됐는데 기존의 AI 학습용 하드웨어 클러스터랑 비교했을 때에이 에이전틱 AI 시대의 플랫폼이 어떻게 밸런스를 유지를 해야 하는지 NBI의 기술적 해법을 좀 더 자세히 들어봤습니다. Agent AI inference is no longer just to generation it is becoming a long workflow involving planning retrable to use code etc. So how does this workload shift change data center GPU platform design compared with traditional training happy GPU clusters and where does the balance shift most across GPU compute CPU orchestration and memory bandwidth and networking?

We have always designed our GPUs to be very high performance for both training as well as inference. You may recall that we were the first uh platform provider to actually pioneer FP4 and specifically NVFP4. NVFP4 is not just for four bit floating point. It is an entire uh format of that includes tensors and scaling factors to absolutely allow you to compress your FP8 parameters and values down to FP4 and minimize your storage use. So we actually look at the entire microchitecture. Where are the areas that we can keep accelerating and every time we identify a new bottleneck, you know, we're following's law throughout the entire AI factory.

As you accelerate one portion of the data center, you end up finding a bottleck somewhere else. So for example, we really accelerated GPUs to the point where we found that they needed more data from storage and from other parts of the data center. As a result, we went into networking and buyed the NET Melanox acquisition. Subsequent to that, we saw CPUs were slowing our GPUs down because they became the next bottleneck. And that's why we started building our own CPUs and now we have the best CPUs when it comes to entic AI workloads.

So as we look across the entire AI factory we identify where is the next bottleneck that we can accelerate because we look at it from a holistic standpoint of the entire workflow. 자, 이렇게 에이전틱 AI 시대가 열리면서 뜻밖에요 CPU 역할이 또 다시금 도마 위에 올랐죠.이 이 AI가 스스로 복잡한 테스크를 제어하고 뭐 파이썬 코드도 실행하고 하드웨어로도 조율를 하고 이런 CPU 사이드의 연산 부담이 굉장히 급증을 했기 때문에 중요해진 건데 이에 대응해서 엔비디아가 에이전틱 AI에 최적화된 새로운 프로세서이 베라 CPU라 거를 이번에 젠슨 키노트에서도 그리고 젠슨 프레스 Q&A에서도 굉장히 강조를 했습니다.이 베라 CPU에 대해서 좀 더 자세히 질문을 드려봤는데요. NBIDIA introduce Vera aspilt for AI agents. In aici.

CPU side workloads such as Python run times orchestration logic code execution are becoming much more important. So from a GPU platform perspective what is the core role of Vera CPU? Is Vera mainly designed to keep GPUs better utilized or should we see it as a new kind of data center CPU built to run agentic workflace more efficiently? So fundamentally Vera is the CPU for the age of entic AI and it has multiple very key innovations. It has basically redesigned the CPU from the groundup for these agentic workflows.

It has very high single threaded single core performance. It has massive memory bandwidth to load and move data around. And it has a scalable coherent fabric that spans and allows you to move data and instructions across the entire die of the CPU. There is no chiplet tax when it comes to moving information around WVER because you're not crossing die to die boundaries. Vera is designed specifically for how do we get data to the core that's going to complete the calculation? How do we complete that very quickly and then how do we send that response back over to the to the GPU?

That's one aspect of Vera. The other aspect is exactly what you just touched on. It's also an orchestration CPU if you will. Because we have the NV link CC. That was another innovation that we introduced. We saw that to keep the GPUs fed with data in a timely and avoid CPU becoming bottleck this aspect of CCV. 자, 반도체 성능을 높이는 거는 단순히 하나의 칩을 이제 만드는 것으로 불가능하고 그래서이 차세대 플랫폼 베라루빈 기준으로 봤을 때는 GPU, CPU 또 인터커넥트 소프트웨어까지 시스템 전체를 하나로 잘 아우를 수 있는 하나의 거대한 컴퓨터처럼 동시에 설계하는 말 그대로 익스트림 코 디자인이라고 하는 극단적인 공동 설계가 정점으로 보이는데요.이 이 에이전트 연상 과정에서 발생하는 수많은 병목 현상을 해결하기 위해서 베라루빈 플랫폼의 선 혁신의 핵심이 무엇인지도 질문을 해 보았습니다.

Next question was about Vera Ruen so Vera Ruben seems to be more than just a GPU generation it looked like AI factory platform could cesigned across GPU CPU networking all the things with like software. In agentic AI a single request can expand into multiple stage of reasoning retrieval to use things like that and making to group more important now. What are the biggest technical contributors to Vera Rubin's performance games and is the biggest difference coming from the Rubin GPU architecture itself or the vera CPU integration Lake fabric memory hierarchy or networking all of them?

It's actually all of them. So you alluded to extreme codesign. We look at the entire workflow, identify where are all the bottlenecks that are going to start appearing. Nvidia is fortunate to have our own inhouse research team uh when it comes to AI models. So we release our own open source models. You may be familiar with Nemotron 3 and Nemotron 3 Ultra. Um we release our own models. So we have a research team that is actively involved in developing state of the art and in where these large language models are going.

As a result of that we can see what needs are coming up on the horizon. When it came to we need to move data quickly across the entire AI factory, we needed a better networking and that's why we started coming up with our own product line there. When it came to these entic workflows, they are going to have multiple calls. In fact, anentic loop can have 160 plus terms where it's repeatedly it's the same as you making 160 rapid prompt inquiries to a model. Being able to hit that, you need to have very rapid GPUs to do all of those computations.

You need very fast CPUs that can actually do validation each of the infront cycles that the GPU is making. So by looking across this entire data center, this entire AI factory, we identified, we looked at law, we looked at what is coming in terms of the workload. How can we identify the need to remove? And then how do we create a product that is highly optimized to improve performance and remove that bottleneck? So you know when competitors say well we have a bigger rack they're not really competing against the NBL72.

They are actually competing against the entire Ruben pod. And the productivity and the performance of that pod far outstrips anything else in the market. It's also in proven benchmarksmitted code repositories. >> 자, 지금이 순간에도 블랙겔이 전 세계에 엄청나게 많이 깔리고 있죠. 그러면서 다음 세대인 베라루빈을 지금 고민을 하고 있는 상황이니까 사실이 K팩스가 엄청나게 지금 투입되고 돈도 더 빌리고 유상증자하고 뭐 여러 가지 일들이 많잖아요. 이런 기존 인프라 투자가 낭비되지가 않고 실제로 돈을 벌려고 하면은 결국 뭔가 블랙에서 베라루빈으로 넘어가는이 전환기 때 세대가 바뀌에 따라서 인프라 연속성과 데이터 아키텍처의 핵심 변화는 무엇인가? 이게 또 실제로 인프라를 사는 입장에서 중요한 포인트일 것 같아서 또 질문을 드렸는데요.

AI factories while also having to think about rubin and future generations. So for them is critical to understand how much of today's GPU infrastructure investment can carry forward. As customer move from Blackwell to V Ruben, what is the most important technical transition from a data center perspective? And which area will be the biggest turning point, a GPU memory or networking fabric? So the really nice thing about working with the Nvidia platform is we always maintain backwards compatibility. CUDA preserved that for over a decade.

The GPU that you buy today will be compatible with any code that you develop for a prior generation. Going from Blackwell to Ver Ruben, we are maintaining that NBL72 Oberon rack architecture. So customers will have a seamless transition if they don't want to change anything else about their AI factory. They can simply move in the Ver Ruben rack into the place of the the black that used to be in that place. There's a minor change in the in the uh the amount of power that's required for the Ver Ruben rack, the NBL, the new NBL72.

But in terms of form factor, deployment, software compatibility all of those things are backwards compa 자, 여기서 또 메모리를 질문 안 할 수가 없죠. 결국 GPU의 연산 속도만큼이나 데이터를 빠르게 주고받고 하는이 메모리 대역폭 밴드위스가 중요하기 때문에 HBM과 관련해서 좀 질문을 드려 봤습니다. From a data center GB perspective, what is the biggest pressure context inference and agent workers place on HBM? Will feature inference performance be driven more by row compet or by architectures that use HPM and other memory movement? So one of the nice things about practicing this philosophy of the extreme codesign is we can actually look at how we can introduce innovation and optimize our needs to what is available from the ecosystem and production capacity as a whole.

So I'll draw your attention to the vera CPU. It actually uses LPDDR and when we first brought that innovation about it uh you know it kind of changes the mentality in the industry but we bring about uh low power to memory. But we also introduce built-in error correction. So you have now enterprise grade memory. It's lower cost and it allows us to bring innovation into a very specific aspect of memory. HPM is going to continue to be important. It's ites right next to the GPU and continuse that we can look across that and >> 이번에 비디아가 키노트에서도 젠슨 CEO가 우리가 진짜 네트워크 최강 회사다 이런 식으로 표현을 하기도 했거든요.

결국 에이전트 AI가 굉장히 긴 장문맥의 어떤 거를 기억을 할 수 있어야 되고 복잡한 인무를 계속 시도를 하려고 하면은 디램에 다 얹지기 힘드니까이 블루필드 4위반의 STX라고 하는이 가속 스토리지 인프라 스토리지라고 하는 거는 여기 지금 랜드 플래시가 들어갔다라는 거죠. 그래서이 데이터 센터에서이 스토리지가 그냥 저장소의 역할이 하는 것이 아니라 장문맥의 어떤 실시간 확장 메모리에 대한 얘기를 하는 것으로 보이기 때문에 여기에 대해서도 한번 물어봤습니다. So for agent AI to perform long and complex tasks the model cannot rely only on the prompt it needs to continuously read long documents pass context to output and enterprise data. Sob has introduced the bluepit 4 SDX as accelerated storage infrastructure for long contest reasoning.

So from a perspective will storage evolve from simply storing data to acting more like context memory for agentic AI the as the agents queries get their tool calling and their requests and the prompt sizes get bigger they will need more access to to fast memory. That's one of the reasons why we introduced the STX rack. Having the Bluefield 4 allows us to do policy management, security, protecting both the data as well as protecting what agents are doing when they are in the data center. Jenson announced our new Doka security product at his keye yesterday and bringing about these innovations allows us to accessible agents latency >> 마지막으로 원래 지난번에는 루빈 CPX라고 GDDR이 들어가 있는이 프리필과 디코드에서 프리필을 책임지는 요런 시스템에 대해서 얘기가 했다가 요즘은 요러한 얘기가 줄어들고 LPX로 바로 전환을 하게 됐잖아요.

요 그러면서 뭐 CPX 이제 앞으로 안 나올 거다. 뭐 이런 얘기도 있고 막 여러 가지 얘기가 많은데 중요한 거는 이렇게 거대하고 이렇게 큰 회사가 이런 의사 결정을 굉장히 민첩하게 하는게 굉장히 전 놀랍더라고요. 그래서 거기에 대해서 실제로이 실무를 하고 계신 분께 NBI 문화라든가 이런 거에 대해서 좀 힌트를 얻을 수 있까 해서 또 질문을 드렸는데요. People often thought training was their hard point while influence was relatively easy the Jesson said and but in the age of the agency AI inf is no longer just simple talk generation it has become a complex systems problem involving planning code execution etc my real question is uh the Rubin CPX uh was originally understood as a dedicated accelerator for the context or previous phase and but in the latest ver Rubin announcement LPX seems to have taken a much more central role.

So should we read this as a sign that Nvidia is not following a fixed chip roadmap but quickly uh reconfiguring CPU GPU and other network LPUs memories interconnects around uh shifting agentic AI bots to optimize the total cost of the AI factory and what is the core capability that allows Nvidia to make this kind of system level architecture pivote so quickly I think it is part of our culture to we we have a very open and transparent culture internally. Teams share a lot of information quickly. We are blessed to have our own does model research on AI models where uh if you look at today we're an AI company but uh at one point in our in our history, we were basically uh we were a chip known as a chip company.

We're the only company that if you look at the semiconductor space that actually creates its own foundation models. Having that knowwhow in house, having this open that allows us to share information, allows us to see where the trends are going, and then teams can come together very quickly and focus on what is next on the road map. We are extremely transparent about what we're doing next. You see our road maps there, public information. We show them in our keynotes over a year in advance. You know what we're thinking of.

But when it comes down to the details of that design and how do you optimize so that the entire workflow is completely seamless, the reason Nvidia is able to make these rapid pivots is just the culture of the company. All these teams are working in concert with each other and that's why we can make these changes on such a rapid manner when we see shifts in the market. Your explanation and so great. 지금까지 에러였습니다. 입니다.