HPC 軟件棧與數(shù)據(jù)分析軟件棧對比

HPC 軟件棧與數(shù)據(jù)分析軟件棧對比

HPC 軟件棧

+------------------------------------------------------------+ Application | Applications and community Codes | Level +------------------------------------------------------------+

MiddleWare +--------------------------------+ +------+ Management | FORTRAN, C, C++, IDE | | | | | | | +--------------------------------+ | | +--------------------------------+ |SCI.V | | Domain Specific library | |IS. | | | | | +--------------------------------+ | | +----------------+ +---------------+ +--------------+| | |MPI/OPENMP +Acce| | Numercial | |Programming/ || | |lerator tools | |Libraries | |Debugging(eg. || | | | | | |PAPI) || | +----------------+ +---------------+ +--------------+| | +----------------+ +---------------+ +--------------+| | |Lustre(Parallel | | Batch Schedu | | System Monit || | |File System) | + ler(eg. SLURM)| | or Tools || | | | | | | || | +----------------+ +---------------+ +--------------++------+ ----------------------------------------------------------------------------------------- System +---------------------------------------------------+ Software | Containers (Singularity, Shifter, etc.) | | | +---------------------------------------------------+ +---------------------------------------------------+ | Linux OS Variant | | | +---------------------------------------------------+ ----------------------------------------------------------------------------------------- Cluster +---------------+ +----------+ +----------+ +-------+ Hardware | Infiniband + | |Local + | | X86 Rack+| |In-Situ| | Eth Switches | | SAN | |GPU or .. | |Proce. | +---------------+ +----------+ +----------+ +-------+

數(shù)據(jù)分析軟件棧

+------------------------------------------------------------+ Application | Manhout R and Applications | Level +------------------------------------------------------------+

MiddleWare+------+ +------------+ +------------+ +--------+ +------+ Management| | | HIVE | | PIG | |FLUME | | | | | | | | | | | | | | | +------------+ +------------+ +--------+ | | | | |Zooke | | | +------------+ +------------+ +------------------+ |eper( | | cloud| |MAPREDUCE | | STORM | | SQOOP | |coordi| |serv | | | | | | | |nation| | | +------------+ +------------+ +------------------+ | | | | +---------------------------------------------------+| | | | | YARN || | | | +---------------------------------------------------+| | | | +---------------------------------------------------+| | | | | HBASE Bigtable(KV Store) || | | | +---------------------------------------------------+ | | +---------------------------------------------------+| | | | | HDFS || | | | +---------------------------------------------------++------+ --------------------------------------------------------------------------------------- System | | +---------------------------------------------------+ Software | | | Virtual Machine and CLoud Service Container | | | | | | | +---------------------------------------------------+ | | +---------------------------------------------------+ | | | Linux OS Variant | +------+ | | +---------------------------------------------------+ ----------------------------------------------------------------------------------------- Cluster +---------------+ +--------------+ +-----------------+ Hardware | Ethenet | | Local Storage| | X86 Racks | | Switches | | | | | +---------------+ +--------------+ +-----------------+

資源管理訴求差異

數(shù)據(jù)格式與文件系統(tǒng)差異

用戶面差異

數(shù)據(jù)挖掘

版權聲明：本文內(nèi)容由網(wǎng)絡用戶投稿，版權歸原作者所有，本站不擁有其著作權，亦不承擔相應法律責任。如果您發(fā)現(xiàn)本站中有涉嫌抄襲或描述失實的內(nèi)容，請聯(lián)系我們jiasou666@gmail.com 處理，核實后本網(wǎng)站將在24小時內(nèi)刪除侵權內(nèi)容。

版權聲明：本文內(nèi)容由網(wǎng)絡用戶投稿，版權歸原作者所有，本站不擁有其著作權，亦不承擔相應法律責任。如果您發(fā)現(xiàn)本站中有涉嫌抄襲或描述失實的內(nèi)容，請聯(lián)系我們jiasou666@gmail.com 處理，核實后本網(wǎng)站將在24小時內(nèi)刪除侵權內(nèi)容。