AMD - AMD MI355X Custom Cluster
MLPerf Inference Category:
datacenter
Availability:
available
Submitted by:
AMD
MLPerf Inference Division:
closed
Accelerator Details
| accelerator_model_name | AMD Instinct MI355X 288GB HBM3e (x87) |
| accelerators_per_node | 8 |
| accelerator_memory_capacity | 288 GB |
| accelerator_host_interconnect | PCIe Gen 5 x16 |
| accelerator_frequency | |
| accelerator_interconnect | XGMI |
| accelerator_interconnect_topology | |
| accelerator_memory_configuration | HBM3E |
| accelerator_on-chip_memories |
Processor and Memory Details
| host_processor_model_name | AMD EPYC 9575F |
| host_processors_per_node | 8 |
| host_processor_core_count | 64 |
| host_processor_frequency | |
| host_memory_capacity | 3.0 TB |
| host_memory_configuration | 24 x 128GB Micron Technology MTC40F2047S1RC64BB1 QSFF |
| host_processor_caches | |
| host_processor_interconnect |
Other Hardware Details
| cooling | Passive & Active |
| disk_controllers | |
| disk_drives | |
| hw_notes | |
| other_hardware | |
| power_management | |
| power_supply_details | |
| power_supply_quantity_and_rating_watts |
Network and Interconnect Details
| host_network_card_count | 4 x 10Gbit/s |
| host_networking | 2 x Ethernet Controller X710 for 10GBASE-T, 2 x Ethernet Controller 10G X550T |
| host_networking_topology | Ethernet on switching network |
| network_speed_mbit | |
| nics_enabled_connected | |
| nics_enabled_firmware | |
| nics_enabled_os | |
| number_of_type_nics_installed |
Software Details
| boot_firmware_version | |
| framework | PyTorch 2.9.1+git8907517, ROCm 7.0.0 |
| management_firmware_version | |
| nics_enabled_firmware | |
| operating_system | Ubuntu 22.04.5 LTS |
| other_software_stack | hipblaslt-1.0.0.70000-38~22.04, vllm-11b6af52, aiter-6af8b687 |
| sw_notes |
Results Table
| Model | Accuracy Target | Server | Interactive | Offline | |||
|---|---|---|---|---|---|---|---|
| Metric | Performance | Metric | Performance | Metric | Performance | ||
| llama2-70b-99 | ROUGE1: 43.9869, ROUGE2: 21.8148, ROUGEL: 28.33, TOKENS_PER_SAMPLE: 265.005 | Tokens/s | 1016380.000 | Tokens/s | 785522.000 | Tokens/s | 1042110.000 |
| llama2-70b-99.9 | ROUGE1: 44.3868, ROUGE2: 22.0132, ROUGEL: 28.5876, TOKENS_PER_SAMPLE: 265.005 | Tokens/s | 1016380.000 | Tokens/s | 785522.000 | Tokens/s | 1042110.000 |