基于SDN的改进延迟调度算法

A Based on SDN Improved Delaying Scheduler

作者: 专业:信息与通信工程 导师:戴彬 年度:2016 学位:硕士  院校: 华中科技大学

Keywords

Cloud computing, Data locality, Delay scheduling, Waiting time threshold, Software defined network

        随着互联网的发展和层出不穷的各种应用,互联网产生着大量的数据,如何有效存储和处理这些大规模数据成为了一个亟待解决的问题。为了解决相关问题,运行大规模分布式计算的数据中心正在全世界范围内构建。作为数据中心核心基础设施的分布式计算框架Hadoop被提了出来。为了减少在作业执行过程中数据迁移所带来的时间开销,在Hadoop中扮演重要角色的作业调度算法尽可能将任务分配到任务所需数据所在的节点上执行,从而缩短了作业响应时间和提高了集群的性能。在保证作业公平性的前提下,由M Zharia等人提出的延迟调度算法尽可能地分配任务到任务所需数据的存储节点。但是,该算法只是根据人工经验设置一个静态等待时间阈值,不能有效适应数据中心动态的负载变化,从而带来调度算法性能的降低。本文所提出的BS-IDS算法充分考虑了影响延迟调度算法性能的数据中心负载情况、网络情况和队首作业的执行状态等几个因素,得到一个自适应的等待时间阈值。通过利用SDN(软件定义网络)对网络控制的灵活性,对数据迁移进行网络控制,避免因网络负载变化而带来的任务响应时间影响。通过实验表明,BS-IDS算法能够有效适应数据中心动态的负载变化,并且相比较于传统的延迟调度算法,在作业响应时间等方面有着更优的表现。
    With the the development of the network and various applications,the internet produces a lot of data,how to store and deal with these large-scale data has become an urgent problem to be solved.In order to meet solve related problems,the operation of large-scale distributed computing data center is being constructed around the world.As the core infrastructure for the data center,the distributed computing framework Hadoop is proposed.For reducing the time caused by data migration during job execution,the algorithm which plays an important role in Hadoop will assign the task to the node where the input data resigns,so as to shorten the response time of the job and improve the performance of the cluster.On the premise of guaranteeing the fairness of the job,the delay scheduling algorithm proposed by Zaharia M el at allocate the tasks to the storage nodes where the data of the tasks require.However,the algorithm just set a static waiting time threshold according to artificial experience,which can not effectively adapt to the dynamic load changes in the data center,leading to the performance of the scheduling algorithm.The proposed algorithm BS-IDS fully consider the data center load,network,the execution state of the head job and so on serveral factors,which affects the performance of delay scheduling algorithm.From this algorithm,we can get an adaptive waiting time threshold. Through the use of SDN(Software Defined Network) to control the flexibility of the network,we can control the network of the data migration,avoiding the impact of the response time casued by changes in the network load.Experimental outcomes indicate that the BS-IDS algorithm is able to adapt to the dynamic change of the data center,and compared with traditional delay scheduling algorithm,it has a better performance in the job response time and so on.
        

基于SDN的改进延迟调度算法

摘要4-5
Abstract5
1 绪论9-17
    1.1 研究背景及意义9-11
    1.2 国内外研究现状11-15
    1.3 本文研究内容及章节安排15-17
2 相关研究技术17-30
    2.1 HADOOP核心技术17-20
    2.2 HADOOP作业调度算法研究20-24
    2.3 SDN核心技术及其发展24-29
    2.4 本章小结29-30
3 基于SDN的改进延迟调度算法30-49
    3.1 方案需求分析30-36
    3.2 基于SDN的HADOOP数据处理框架36-38
    3.3 基于SDN的改进延迟调度算法38-48
    3.4 本章小结48-49
4 BS-IDS算法的设计与实现49-62
    4.1 作业执行流程49-51
    4.2 BS-IDS调度器的总体设计51-54
    4.3 BS-IDS调度器各模块设计与实现54-61
    4.4 本章小结61-62
5 实验与测试结果分析62-70
    5.1 实验环境搭建与参数配置62-63
    5.2 实验测试结果分析63-69
    5.3 本章小结69-70
6 全文总结及展望70-72
    6.1 总结70
    6.2 展望70-72
致谢72-73
参考文献73-77
        下载全文需58


本文地址:

上一篇:自组织网络中调度算法的研究
下一篇:面向数据中心网络的SDN控制器设计与实现

分享到: 分享基于SDN的改进延迟调度算法到腾讯微博           收藏
评论排行