Top suggestions for Collective Communication AlgorithmsExplore more searches like Collective Communication AlgorithmsPeople interested in Collective Communication Algorithms also searched for |
Top suggestions for Collective Communication AlgorithmsExplore more searches like Collective Communication AlgorithmsPeople interested in Collective Communication Algorithms also searched for |
Feedback
+and+high+performance+computing+(HPC)+workloads%252C+and+optimizing+its+performance+is+of+interest+to+both+ML+and+HPC+communities.+All-to-all+is+a+particularly+challenging+workload+that+can+severely+strain+the+underlying+interconnect+bandwidth+at+scale.+This+paper+takes+a+holistic+approach+to+optimize+the+performance+of+all-to-all+collective+communications+on+supercomputer-scale+direct-connect+interconnects.+We+address+several+algorithmic+and+practical+challenges+in+developing+efficient+and+bandwidth-optimal+all-to-all+schedules+for+any+topology+and+lowering+the+schedules+to+various+runtimes+and+interconnect+technologies.+We+also+propose+a+novel+topology+that+delivers+near-optimal+all-to-all+performance.&cdnurl=https%3a%2f%2fth.bing.com%2fth%2fid%2fR.4be08737a19782f2fe5117e03f861fb3%3frik%3dsF7oit48ijyxyA%26pid%3dImgRaw%26r%3d0&exph=630&expw=1200&q=Collective+Communication+Algorithms&FORM=IRPRST&ck=56819F32F37B96681DCE6D157C7B7BFD&selectedIndex=32&itb=0)
